Re: VMInterface addition: Make native library names part ofVMInterface
David I don't think there is an easy solution to this as it is unlikely David that a single VMInterface will fit the needs of all VMs perfectly. David In some cases (java.lang.ref.* for example), I don't think that it David is reasonable for classpath to try to provide an implementation David that will actually work unchanged on all VMs. Andrew Exactly. Classpath should provide a reference implementation, Andrew we but shouldn't expect every core class to be used unchanged. I agree. I think it is reasonable to expect libgcj to diverge from classpath in some places -- there are areas where libgcj's needs aren't the same as the needs of other VMs. But at the same time, I'd expect VMs that are unusual in other ways (e.g., IKVM or JikesRVM, both different in one way or another) to also occasionally have their own divergence. Or, to put it another way, let's distribute some of the divergence; it can't all rest on libgcj. (Which I doubt anyone is proposing anyway.) Tom ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
I agree. disclaimer: I am not a lawyerso my grasp on the fine details of licensing is not to be trusted One issue that comes up for Jikes RVM is that because we are licensed under the CPL we believe it is hard for us to provide modified versions of classpath in a way that is easy for our users to deal with. Because of this we have taken a tact of either (1) using a classpath class unchanged or (2) providing a complete replacement for the class. In most cases this is ok, but there are some cases where we want/need to make a very small change to a classpath class and things get awkward for us. In many cases, this can be done by adding a VMFoo class and a level of method calls. For us, this introduces no overhead because we inline them all away. It seems to me that the key question is can we construct this stuff in such a way that it causes no overhead for gcj as well (is there some stylized cross class inlining that you can do easily). If so, then there is a chance to really reduce the divergences by writing much of the sensitive code in this fashion, If not, then it is harder to balance the needs of the various VMs vs the obvious goodness of having as close to a single unified code base as possible. I personally really don't like having our own copies of entire classes because it is harder to track changes in classpath and benefit from bug fixes. So, there are places where Jikes RVM is living with very inefficient implementations of library functionality (almost everything having to do with stack inspection except java.lang.Throwable which we did reimplement) because in most cases the pain of diverging has been more important than the performance cost. --dave Tom Tromey [EMAIL PROTECTED] 11/07/2003 02:47 PM Please respond to tromey To: Andrew Haley [EMAIL PROTECTED] cc: David P Grove/Watson/[EMAIL PROTECTED], GNU Classpath [EMAIL PROTECTED] Subject: Re: VMInterface addition: Make native library namespart ofVMInterface David I don't think there is an easy solution to this as it is unlikely David that a single VMInterface will fit the needs of all VMs perfectly. David In some cases (java.lang.ref.* for example), I don't think that it David is reasonable for classpath to try to provide an implementation David that will actually work unchanged on all VMs. Andrew Exactly. Classpath should provide a reference implementation, Andrew we but shouldn't expect every core class to be used unchanged. I agree. I think it is reasonable to expect libgcj to diverge from classpath in some places -- there are areas where libgcj's needs aren't the same as the needs of other VMs. But at the same time, I'd expect VMs that are unusual in other ways (e.g., IKVM or JikesRVM, both different in one way or another) to also occasionally have their own divergence. Or, to put it another way, let's distribute some of the divergence; it can't all rest on libgcj. (Which I doubt anyone is proposing anyway.) Tom ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
- Original Message - Tom Tromey writes: David I don't think there is an easy solution to this as it is unlikely David that a single VMInterface will fit the needs of all VMs perfectly. David In some cases (java.lang.ref.* for example), I don't think that it David is reasonable for classpath to try to provide an implementation David that will actually work unchanged on all VMs. Andrew Exactly. Classpath should provide a reference implementation, Andrew we but shouldn't expect every core class to be used unchanged. I agree. I think it is reasonable to expect libgcj to diverge from classpath in some places -- there are areas where libgcj's needs aren't the same as the needs of other VMs. But at the same time, I'd expect VMs that are unusual in other ways (e.g., IKVM or JikesRVM, both different in one way or another) to also occasionally have their own divergence. Or, to put it another way, let's distribute some of the divergence; it can't all rest on libgcj. (Which I doubt anyone is proposing anyway.) I agree with you. My approach is rather pragmatic: first, classpath should allow to quickly develop a VM, by providing a clean design and as many features as possible in Java. Then, once the VM is running, everybody can choose the optimizations that match the problem that the VM is trying to address. In practice, as soon as one VM leaves the state of pet project or proof of concept, optimizations are unavoidable. I'm not sure that compiler optimizations (in particular inlining and pointer escape analysis) can handle every problem (and as I remember, they ofter require static compilation and limit dynamic loading, a core java feature), thus some code modification may be required to optimize the API for the VM. Anyway, coming myself from the compiler world, you can imagine how painful it is for me to say that native methods should forward the call to the VM* classes (thus adding an indirection). This has for me the following advantages: * clear separation between the API and the VM * regrouping the native methods into a few classes (it is simpler to find out which methods must be implemented, and the number of stubs to create is much smaller, which is less code to manage) On the other hand, the VM* approach causes additional costs: * often requires to duplicate structures to keep track of one information. * additional indirection in each invocation As often the case, good design and optimizations don't mix well! In case of doubt, I always choose the good design (it fits every case) over the optimization (it usually fits only a specific use-case). But this is just my own opinion. -Patrik Patrik Reali http://www.reali.ch/~patrik/ http://www.oberon.ethz.ch/jaos ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
Hi, On Wed, 2003-11-05 at 21:31, Bryce McKinlay wrote: On Nov 5, 2003, at 5:56 AM, Jeroen Frijters wrote: I think that is in fact what Mark was suggesting and I think this is definitely a good idea. There are a lot of VMs that don't (want to) use JNI for their native methods. Having all native methods in the VM* classes makes this much easier. I was suggesting that. Sorry for not responding earlier. But having you and Patrik affirm that it is a good thing is more valuable then when 'the guy without his own VM' says it :) I think its a mistake for classpath to try to be everything for everyone, even at the expense of elegance and efficiency. At some point, most implementations are going to have to modify it in some way in order to suit their special/custom needs. Hopefully we won't do it at the expense of elegance 'and' efficiency, but I agree that sometimes it won't be possible to achieve both. As our Hacker Guide puts it (even though it leaves out elegance): When you write code for Classpath, write with three things in mind, and in the following order: portability, robustness, and efficiency. If efficiency breaks portability or robustness, then don't do it the efficient way. If robustness breaks portability, then bye-bye robust code. Of course, as a programmer you would probably like to find sneaky ways to get around the issue so that your code can be all three ... And I am also sure that there might be better designs then the grouping of platform dependent methods into their own VMInterface classes for some VMs. But just marking some methods 'native' to indicate that they are platform dependent is not it. A more efficient (and probably more flexible) approach might be to introduce some preprocessor that munges the java source files. But doing that in a clean way will be tricky. One of the big plus points of defining all platform dependent methods in VMInterface classes is that it makes clear when you are about to break Classpath any VM, so you must explicitly document your change. And such documenting and clear marking of 'special' VM classes makes it a lot easier for VMs that don't follow classpath CVS all the time, but just import a new version when we create a new snapshot release. What I hoped was that by having the VMInterfaces classes clearly marked as 'special', by being package local, final, having (mostly) static methods, and starting the name with VM (*), would make it easy for VMs and compilers to just inline all call to them. Especially if you treat the Classpath core classes as 'sealed' and compile them as complete packages. Maybe this is much harder then I imagined. What would be very beneficial would be if we took the script that gcj uses to display the changes between current Classpath CVS and gcc CVS and apply it to all current VMs that use a modified Classpath. http://gcc.gnu.org/java/libgcj-classpath-compare.html It has a way to mark changes as 'local' to the VM implementation which would be very helpful for all Classpath/VM integrators. And if more VMs used it then it gives the Classpath developers a better clue about what kind of things have to be adapted for every VM anyway. I was pleased to hear at the GNU Classpath workshop that at least Kissme, IKVM and Jaos use an unmodified GNU Classpath as core library implementation (and I believe JikesRVM also uses a unmodified Classpath). So we are not doing that bad. But maybe those VM/Compilers are not as aggressive about efficiency and elegance as gcj is (which does produce some very fast code!). But I still like to hear how we can improve the VM/Compiler/Platform specific parts of GNU Classpath without making life much harder for the gcj hackers. Cheers, Mark (*) Maybe VMInterface was wrong and we should actually call these classes Platform_Interface classes. signature.asc Description: This is a digitally signed message part ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
Mark Wielaard writes: Hopefully we won't do it at the expense of elegance 'and' efficiency, but I agree that sometimes it won't be possible to achieve both. As our Hacker Guide puts it (even though it leaves out elegance): When you write code for Classpath, write with three things in mind, and in the following order: portability, robustness, and efficiency. If efficiency breaks portability or robustness, then don't do it the efficient way. If robustness breaks portability, then bye-bye robust code. Of course, as a programmer you would probably like to find sneaky ways to get around the issue so that your code can be all three ... And clearly these guidelines are at odds with those of libgcj, where (and this is just my opinion) we favour robustness, maintainability, efficiency, portability. My goal is to improve gcj's speed to the point where it matches all of its competitors. I will be very cross indeed if while I am improving the compiler someone is concurrently pessimizing the library, What I hoped was that by having the VMInterfaces classes clearly marked as 'special', by being package local, final, having (mostly) static methods, and starting the name with VM (*), would make it easy for VMs and compilers to just inline all call to them. Especially if you treat the Classpath core classes as 'sealed' and compile them as complete packages. Maybe this is much harder then I imagined. It's not particularly difficult to do, but at the moment we don't do it. gcj is perfectly capable of whole-program Java optimization but we have to follow Java rules, and one of those rules is that you cannot inline final methods because it breaks binary compatibility. We could teach gcj that VMfoo classes are an exception to that rule, but I think you would agree that's not a pretty solution. Alternatively, we could perhaps have a compiler switch that allows final methods to be inlined, so gcj -finline-final foo.class VMfoo.class inlines all calls to final methods between foo and VMfoo. However, this would force us to rewrite the libgcj build scripts to handle VMfoo classes in a special way. However, none of this addresses Bryce's valid point that too much abstraction increases the complexity of the classes, making maintenance more difficult. I've worked on systems with too much abstraction. They are very hard to understand and debug. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
On Thursday 06 November 2003 12:04, Sascha Brawer wrote: Andrew Haley [EMAIL PROTECTED] wrote on Thu, 6 Nov 2003 10:28:24 +: [having the VMInterfaces classes clearly marked as 'special' would make it easy for VMs and compilers to just inline all calls to them] It's not particularly difficult to do, but at the moment we don't do it. gcj is perfectly capable of whole-program Java optimization but we have to follow Java rules, and one of those rules is that you cannot inline final methods because it breaks binary compatibility. We could teach gcj that VMfoo classes are an exception to that rule, but I think you would agree that's not a pretty solution. Apparently, Java 1.5 will have meta-data (arbitrary, uninterpreted declarations for language constructs that are represented in bytecode and accessible via reflection), similar to CLI/.NET. This might be a much cleaner approach for declaring such exceptions. Of course, the free compilers would need to support it first. You could do it already using the Attributes mechanism - Java implementations are supposed to silently ignore attributes they don't understand. The attributes could be inserted by a tool using BCEL, so you wouldn't actually need to modify the compiler ... -- Chris Gray/k/ Embedded Java Solutions Embedded Mobile Java, OSGi http://www.kiffer.be/k/ [EMAIL PROTECTED] +32 477 599 703 ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
Sascha Brawer writes:
Andrew Haley [EMAIL PROTECTED] wrote on Thu, 6 Nov 2003 10:28:24 +:
[having the VMInterfaces classes clearly marked as 'special'
would make it easy for VMs and compilers to just inline all calls
to them]
It's not particularly difficult to do, but at the moment we don't do
it. gcj is perfectly capable of whole-program Java optimization but
we have to follow Java rules, and one of those rules is that you
cannot inline final methods because it breaks binary compatibility.
We could teach gcj that VMfoo classes are an exception to that rule,
but I think you would agree that's not a pretty solution.
Apparently, Java 1.5 will have meta-data (arbitrary, uninterpreted
declarations for language constructs that are represented in bytecode and
accessible via reflection), similar to CLI/.NET.
So we'd have something like
public final class VMfoo
{
@Inline void native bar (int i);
@Inline void baz (int i) {
I could make that work.
Andrew.
___
Classpath mailing list
[EMAIL PROTECTED]
http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
Jikes RVM uses a mostly unmodifed classpath. We don't require users to patch any classpath sources, but there are currently 12 non-VM classes for which we provide our own implementation. java.lang.ref: PhantomReference, Reference, SoftReference, WeakReference java.lang: Class, Object, Runtime, Thread, Throwable java.lang.reflect: Constructor, Field, Method Some of this is historical (we had the classes already and haven't had the bandwidth to try to integrate our versions with classpath). In the case of java.lang.ref.* we really needed to do something different than what classpath provided, and I don't see that changing (and the classes are tiny anyways). There may be hope for merging some of the java.lang and java.lang.reflect classes eventually. Jikes RVM cares a lot about efficiency, but we have different constraints than gcj. It's very easy for us to inline across classes without breaking any semantics. The VM itself is also written in Java (with some extensions to allow the VM to do type unsafe things like implement a GC), so for us efficiency usually means avoiding native methods and staying in Java+VM_Magic (this is what motivated the VMFloat/VMDouble changes I contributed to classpath a while back...Jikes RVM can do these operations much more efficiently by staying in Java instead of calling a native method). But, an extra level of final methods is harmless for us, we inline it all away. --dave___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
Mark Wielaard wrote: I think that is in fact what Mark was suggesting and I think this is definitely a good idea. There are a lot of VMs that don't (want to) use JNI for their native methods. Having all native methods in the VM* classes makes this much easier. I was suggesting that. Sorry for not responding earlier. But having you and Patrik affirm that it is a good thing is more valuable then when 'the guy without his own VM' says it :) I am also strongly in favor of putting all VM-specific native methods in VM* classes, and all library-specific native methods outside of VM* classes. It even helps for code maintenance, and you can then use automated tools to look for missing native implementations in the VM very easily. Same applies to class library. Hopefully we won't do it at the expense of elegance 'and' efficiency, but I agree that sometimes it won't be possible to achieve both. As our Hacker Guide puts it (even though it leaves out elegance): When you write code for Classpath, write with three things in mind, and in the following order: portability, robustness, and efficiency. If efficiency breaks portability or robustness, then don't do it the efficient way. If robustness breaks portability, then bye-bye robust code. Of course, as a programmer you would probably like to find sneaky ways to get around the issue so that your code can be all three ... And I am also sure that there might be better designs then the grouping of platform dependent methods into their own VMInterface classes for some VMs. I habve yet to find time for it, but I should within the upcoming months. I see the possibility to use m4-like macros so that most VMs would share VM*.java implementations, but be able to apply changes to them without disrupting other VMs. Etienne -- Etienne M. Gagnon, Ph.D. http://www.info.uqam.ca/~egagnon/ SableVM: http://www.sablevm.org/ SableCC: http://www.sablecc.org/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
I am also strongly in favor of putting all VM-specific native methods in VM* classes, and all library-specific native methods outside of VM* classes. I suspect the notion of a VM-specific native method vs. a library-specific native method is pretty fuzzy. One example is VMFloat, most VM's would probably view those as library specific, but Jikes RVM wants to think of them as VM-specific because we can do a better job of implementing them in a VM-specific way than the native method does (and we don't want to have to provide our own complete implementation of Float.java just to change a couple methods). But, the native method is VM neutral and works just fine on Jikes RVM, it is just much less efficient for us. There are also obviously Java methods that are (or can be) VM specific. Some of these have been moved to VM* classes, others have not. I don't think there is an easy solution to this as it is unlikely that a single VMInterface will fit the needs of all VMs perfectly. In some cases (java.lang.ref.* for example), I don't think that it is reasonable for classpath to try to provide an implementation that will actually work unchanged on all VMs. The details of this class are pretty closely tied into the details of the GC system and there is very little code that isn't VM specific. --dave___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
David P Grove writes: I don't think there is an easy solution to this as it is unlikely that a single VMInterface will fit the needs of all VMs perfectly. In some cases (java.lang.ref.* for example), I don't think that it is reasonable for classpath to try to provide an implementation that will actually work unchanged on all VMs. Exactly. Classpath should provide a reference implementation, we but shouldn't expect every core class to be used unchanged. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
Patrik Reali writes: Jeroen wrote: Bryce McKinlay wrote: Sorry, I think I misunderstood your message. I thought you were suggesting moving all the native methods (eg for IO classes) to separate VM* classes. I think that is in fact what Mark was suggesting and I think this is definitely a good idea. There are a lot of VMs that don't (want to) use JNI for their native methods. Having all native methods in the VM* classes makes this much easier. I agree with this. Moving all the native methods in the VM* classes would be easier for the VM implementors, because it is easier to document or find out which methods must be provided (a grep native *.java doesn't help much!). Well, it depends. A downside is that this will cause divergewnce between libgcj and classpath, because of course we won't add another indirection where it isn't necessary. We don't have global inlining in gcj at the moment, and two trips through the PLT for simple native method invocations would be intolerable. I suppose we could put some sort of evil hack into the compiler to redirect such calls to the real implementation, but I don't like that idea much. Yea, we will eventually have global inlining so we can redirect trivial calls. However, in the presence of inheritance it's not always easy to do this: binary compatibility says that even a call to a final method can't necessarily be inlined because the target method might be made non-final. Andrew. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
On Nov 5, 2003, at 5:56 AM, Jeroen Frijters wrote: Bryce McKinlay wrote: Sorry, I think I misunderstood your message. I thought you were suggesting moving all the native methods (eg for IO classes) to separate VM* classes. I think that is in fact what Mark was suggesting and I think this is definitely a good idea. There are a lot of VMs that don't (want to) use JNI for their native methods. Having all native methods in the VM* classes makes this much easier. On the other hand, too much abstraction increases the complexity of the classes, making maintenance more difficult. Even putting efficiency issues aside, I agree with Andrew - we would not want to do this in libgcj. This assumption is not true for some VMs. My VM (IKVM) has no native methods and I'm pretty sure this is also true for JAOS and maybe others. I think its a mistake for classpath to try to be everything for everyone, even at the expense of elegance and efficiency. At some point, most implementations are going to have to modify it in some way in order to suit their special/custom needs. Regards Bryce. ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
RE: VMInterface addition: Make native library names part ofVMInterface
Bryce McKinlay wrote: Sorry, I think I misunderstood your message. I thought you were suggesting moving all the native methods (eg for IO classes) to separate VM* classes. I think that is in fact what Mark was suggesting and I think this is definitely a good idea. There are a lot of VMs that don't (want to) use JNI for their native methods. Having all native methods in the VM* classes makes this much easier. Right. I think there is a distinction, however, between what the VM must implement to operate with classpath - ie core VM classes like Class, Object, Throwable, Thread; and portable classes which happen to have native methods, such as java.io.File and java.net.PlainSocketImpl. The later are just normal classes with native methods, the implementations of which are typically be portable across different VMs. This assumption is not true for some VMs. My VM (IKVM) has no native methods and I'm pretty sure this is also true for JAOS and maybe others. So, they a system/platform interface rather than the VM interface. To put it another way, just because a method is native doesn't mean it interfaces with the VM. The VM* classes don't mean interfaces with the VM, but are a way for VM implementers to easily replace their implementation. (The idea being that the interface between the non-VM and VM classes is fairly stable). Regards, Jeroen ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
Re: VMInterface addition: Make native library names part ofVMInterface
Jeroen wrote: Bryce McKinlay wrote: Sorry, I think I misunderstood your message. I thought you were suggesting moving all the native methods (eg for IO classes) to separate VM* classes. I think that is in fact what Mark was suggesting and I think this is definitely a good idea. There are a lot of VMs that don't (want to) use JNI for their native methods. Having all native methods in the VM* classes makes this much easier. I agree with this. Moving all the native methods in the VM* classes would be easier for the VM implementors, because it is easier to document or find out which methods must be provided (a grep native *.java doesn't help much!). Right. I think there is a distinction, however, between what the VM must implement to operate with classpath - ie core VM classes like Class, Object, Throwable, Thread; and portable classes which happen to have native methods, such as java.io.File and java.net.PlainSocketImpl. The later are just normal classes with native methods, the implementations of which are typically be portable across different VMs. This assumption is not true for some VMs. My VM (IKVM) has no native methods and I'm pretty sure this is also true for JAOS and maybe others. How I'm going to explain this...? Well Jaos uses the Oberon language to provide the native methods (native == written in a language other than java), but doesn't go through JNI: both languages have the same object model with garbage collection, so no object pinning and similar stuff is needed, and the jitter compiles the byte-code using the same data layout and calling convention, such that the generated code is identical for both languages. It is the task of the linker to patch the native implementation entry points into the method tables, and from that point java can call oberon and oberon can call java. Moving all the native methods to a separate class would make my job simpler: I would replace the complete class with an oberon class instead of having to merge them. And because the interface is stable, I would be able to get rid of a lot of bootstrap code, which checks every time whether the methods and fields have changed. So, they a system/platform interface rather than the VM interface. To put it another way, just because a method is native doesn't mean it interfaces with the VM. The VM* classes don't mean interfaces with the VM, but are a way for VM implementers to easily replace their implementation. (The idea being that the interface between the non-VM and VM classes is fairly stable). Well, the VM classes are part of the VM interface, which is much broader. I agree that they will be fairly stable, thus making the port simpler to maintain. -Patrik Patrik Reali http://www.reali.ch/~patrik/ http://www.oberon.ethz.ch/jaos/ ___ Classpath mailing list [EMAIL PROTECTED] http://mail.gnu.org/mailman/listinfo/classpath
