JVM Spec version 2 and .class file versioning
While reviewing the class file structure of java.lang.Object in Sun's 1.4.2_06 JDK, I noticed that its major_version field is 48. The JVM Spec version 2 says, Implementations of version 1.2 of the Java 2 platform can support class file formats of versions in the range 45.0 through 46.0 inclusive. (That is, major.minor version fields in the class file.) The 1.5.0 JDK docs includes the same JVM spec version as does the 1.4.2, namely at http://java.sun.com/docs/books/vmspec/2nd-edition/html/VMSpecTOC.doc.html for both of them. Yet the root object file java.lang.Object has minor/major version fields set to 0/48, respectively, thus version 48.0, which is outside of the acceptable range for both JDK 1.4.2 and 1.5.0. Upon further inspection of this .class file (see also JVM spec section 4.1 et al), I noticed that following the access_flags field, there appears to be 8 unknown bytes, followed by an expected three words of zeroes for the this_class, super_class, and interfaces_count fields. What follows those, though, is not intelligible to me via the spec. What gives? Where can I find correct descriptions of the latest .class file definitions for JDK 5? Dan Lydick
Re: [arch] How much of java.* and friends does Harmony need towrite. Was: VM/Classlibrary interface
[Original Message] From: Sven de Marothy [EMAIL PROTECTED] To: harmony-dev@incubator.apache.org Date: 6/4/05 3:49:37 PM Subject: Re: [arch] How much of java.* and friends does Harmony need towrite. Was: VM/Classlibrary interface On Sat, 2005-06-04 at 21:44 +0200, Santiago Gala wrote: So why create flexibility when there aren't options? to enable the development of other options? without arriving to flexibility syndrome, good interfaces enable competing implementations. If you're going to go off and write your own class library, implementing the VM-specific parts is the least of your problems. Your question has exactly the same mindset that Sun people's comments: Java is free enough, you have our implementation for free, why do you want another one? :-) No, it isn't the same mindset at all, because I can give you a specific list of grievances I have with the Sun license. But nobody here seems to be able to give list any specific problems with the GNU Classpath VM interface other than It's not ours.. So if it's available, let Geir and co. keep working on any legalities with FSF for using it to the max. That's what they wrote it for, and I'm all for investigating its potential. /Sven Dan Lydick
Re: Summer of Code, something for Harmony?
We don't necessarily have to do it here, but given the wide and deep community here, we might be able to get some benefit through coordination and networking. geir On Jun 1, 2005, at 9:11 PM, Leo Simons wrote: On 01-06-2005 18:27, Geir Magnusson Jr. [EMAIL PROTECTED] wrote: I thought about it. Me too. I'm not sure we'd have a project for someone, but if we do, it's a good thing for us to do. I'm not sure, but I don't really think so. Harmony has a lot to figure out for itself (there's not exactly a very experienced stable community yet with a lot of leftover capacity for teaching open source community skills) before taking this on. I would encourage people who like this idea in general to approach google whether projects related to open source java could be tackled under a different banner (ie like hacking on Classpath or jikes or gcj or Kaffe or ...). if not, I still urge anyone that has the time to help out in whatever way they can. Aye! Leo -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] VM/Classlibrary interface
On Jun 3, 2005, at 7:18 AM, Peter Edworthy wrote: Hello, And you can circumvent the language protection (package private...) if you work hard enough too, I believe... Keeping out of java.lang seems wise if we can arrange it... I agree, but ClassPath has its interface classes in Java.lang and while we can and probably should implement these as an adapter level to our internal system abstraction I can see why the JVM providing # java.lang.Class # java.lang.Runtime # java.lang.Thread # java.lang.reflect.Constructor # java.lang.reflect.Method # java.lang.reflect.Field makes sense, as these are completely dependant on information only available to the JVM. ClassPath could implement these objects and then have them call across to, say, jvm.implementation.* objects but this adds an extra level of indirection with no increase in flexibility of the implementation. Oops. I see that I wasn't clear. I didn't meant that the VM shouldn't implement things from java.lang - it makes perfect sense that we do so. What I was trying to say [sloppily] was that we shouldn't extend java.lang w/ new classes or interfaces. Sorry. The other java.lang.VM* objects I agree should really not be in Java.lang.* but having references to these use an explicit ClassLoader mitigates this into a cosmetic issue. Yes It seems to me that control of 'java.lang' and 'java.lang.*' would be the simplest way to deal with issues of control of the implementation so that it is a Harmony implementation, not one from Sun, IBM, FSF, Sourceforge.* or anyone else. From the Sun JDK 1.5.0 src.zip, there are 49 distinct 'sun.*' imports in the 'java.lang' package, none of which are found in this archive: I wasn't aware that we were trying to restrict the possible ClassLibs that could be used. In an ideal world we would want to make switching between ClassPath and Jikes's implementations easy, encouraging the user to use the most efficient implementation. (see next email for suggestions of mix and match implementations). yes, and it would be cool if we could concoct an adapter for Sun's implementation as well for fun :) Making the interfaces between VM and class library standard should be something we do now, starting with GNU Classpath's interface, and figuring out what else we might need based on experiences of others coughIBM/Intel/BEA/cough doing the same thing. ...snip... By writing _only_ java.lang and java.lang.*, we can truly speak of a separate implementation. Adding only _parts_ of libraries like GNU ClassPath would mean that users would implement Harmony library policies, not Sun's, FSF's, or anybody elses. This implemetation of java.lang and java.lang.* is only 165 classes in 53K lines of code, not including native code. Doable by this group without a doubt. Even better than that the ClassPath project has already done this. As I understand it, we are discussing methods of interfacing with the ClassPath project in such a way that; other implementations can be plugged in with minimal coding, changes to the Java API do not require changes to the JVM, only the ClassLibs. Well, two comments : 1) we clearly want to interface w/ GNU Classpath, but we want to generalize thought whatever we can learn from others. 2) I don't know for sure, but guess that as time goes on and the Java API evolves, it might require additions to the JVM/Classlib API geir Thanks, Peter -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] VM/Classlibrary interface
On Jun 3, 2005, at 1:06 PM, Sven de Marothy wrote: Hello, And you can circumvent the language protection (package private...) if you work hard enough too, I believe... Keeping out of java.lang seems wise if we can arrange it... By writing _only_ java.lang and java.lang.*, we can truly speak of a separate implementation. Adding only _parts_ of libraries like GNU ClassPath would mean that users would implement Harmony library policies, not Sun's, FSF's, or anybody elses. Blatant Not-Invented-Hereism. This isn't about Control over the implementation, this was about language protection features. I won't comment on NIH (as I didn't quite grok what Dan was saying - I'm at 37k feet coming back on a redeye...) but we need to figure out if the language protection features are enough of a benefit to outweigh he risks of extending java.lang, and if the needs of 1.5 require so much more that the language protection is moot. The FSF doesn't have any library policy. What are you talking about? By writing _only_ java.lang and java.lang.*, we can truly speak of a separate implementation. Why do you need a seperate implementation? Separate from GNU Classpath or something else? geir /Sven -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Jun 3, 2005, at 6:39 PM, Dalibor Topic wrote: That's a rather weird thing to standardise, being the internals of a binding between the library and the VM, and necessarily something that is in flux, as VMs and class libraries change internally. And they do change quite a bit over time ;) I assume they only change as the Java API changes? I.e. the interface needed for 1.4 could be different than the interface needed for 1.5 etc? (and similarly, if the interface is different, it implies a required change on both sides of it ?) If that's so, then I think it is reasonable to standardize - you'd just have the standard interface for a given Java API version... geir -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Jun 4, 2005, at 12:59 PM, Sven de Marothy wrote: On Fri, 2005-06-03 at 14:01 -0500, Dan Lydick wrote: Naw, but have you ever looked into how to design and construct a JVM? The fundamental classes like java.lang can typically have implementation-specific requirements, so I am trying to focus on isolating these items from the rest of the library. Right, this is a concern for all. GNU Classpath does this through its VM inteface classes (e.g. VMObject, VMClass, VMClassLoader) I don't see why this isn't good enough. It's certainly seems good enough for the existing VMs which use Classpath. What I mean is implementation policy on how a class library does its work. If the Harmony implementation can keep from being forced to do things somebody else's way, then Harmony may use libraries from vendors such as these without concern of being forced into their JVM or other class library implementation. Basically this means commanding a central core of packages via the bootstrap class loader and letting a library supplier do the rest. Well, again, I can't see what's so bad about Classpath's way of doing this. And I can't see why you would want this freedom. Freedom is good! Software livre! (I *just* couldn't resist...) AFAIK there are no other class libraries out there which you'll legally be able to distribute with Harmony. So why create flexibility when there aren't options? Are you kidding? There aren't options *now* (well, that's not really true, is it...), but that doesn't meant that options won't come around in the future. I think we're still in the very beginning of managed runtime environments and generalization w/o penalty is a Good Thing(tm). I mean, you can at least just use the Classpath interface for the time being, and use this strategy once there is some reason to. True, except I really hate extending java.lang. :) And maybe we have more to learn in this area from other implementations and newer Java APIs. The underlying idea here is to make as few changes as possible to as little of the java.*, especially java.lang.*, or other core library packages in order to give the Harmony JVM runtime environment the greatest flexibility for using libraries. Heck, if it's done right, you might be able to use Sun's or IBM's java.* library implementation! Why would you want to have a Free VM which can use non-free libraries? why not? Why restrict that freedom for users? Why would anyone want to do that? You can't distribute them together. I think that depends upon your definition of free. If you mean free as allowing unhindered action, then you could. If you mean free as in bound by restrictions on the recipient that prevents choice in action, then no, you are right. Remember, we (at the ASF) don't mind that someone might take our software and bundle with proprietary anything. We think that the market drives contributions back - it doesn't need to be forced through a license. Really, if you want a real solution here, it's to get Sun to publish a spec for the VM-Classlib interface which we can all use, and this problem will go away by itself. With input from Sun, IBM, BEA, HP, GNU Classpath and everyone else w/ experience here, we have a chance of achieving just that. Sun is a welcome and peer participant in this community, and I think they have a lot to offer, if they'd just start contributing hint, hint. At least this is my idea. I don't know if this is actually possible because it is heavily dependent on the library implementation from vendor X, Y, and Z. I do like the idea of using/reusing GNU Classpath where it shines and of Harmony either contributing to it or extending it where some improvements are appropriate or writing complete replacements where the implementation is too weak for our use. At least this is what I have gathered from others in the discussion on the list on this subject. The way I've intepreted most of the posts here, is that most were decidedly against forking Classpath. What makes you think that there are Harmony-specific improvements to be made which wouldn't be usable by others? I feel like there's a lot of uncertainty being cast on GNU Classpath here for no reason. A lot of folks seem to have the impression we've got different goals and/or priorities. We do not. I don't believe we do. I personally have engineering concerns about the interface (distinguished from legal/license or philosophical concerns, both of which I left behind when we started this...) I don't want to fork GNU Classpath, and I want to do everything I can to help that community flourish. That said, I do want to keep the door open for others. This is the extent of what I mean. I don't want to re-invent any wheels that don't need it. Ok. Well I still don't understand. Classpath has a VM-classlib interface which is being used by a whole bunch of VMs. If that
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Jun 3, 2005, at 5:22 PM, Aaron Hamid wrote: I'm sort of confused by this discussion also. It seems to be about at what granularity we start introducing vm- specific interfaces/implementations. At some point the class library has to interface with the VM. We can do that at a very high level of course, by simply subsuming the entirety of java.lang.* and avoid this issue by simply making the java.lang package THE interface, Well, if GNU Classpath is any indication, you *can't* do that w/o extending java.lang, and therefore you aren't avoiding the issue - you are just burying it in java.lang. but we certainly don't want to rewrite and be responsible for all that code if we can avoid it. The only other alternative (at least that I have read) is some compromise whereby core classes rely on an even-more-core VM-specific API that will necessarily be different for each VM. Yes - and that would be opaque to the class library implementations... for example, the interface contract would specify what is needed for both sides (VM and class lib) and then it's up to the implementor of the VM to do the right thing wrt the interface w/ the full freedom to do it in any way they want, and similar for the class library implementation. As far as I can tell, there is no such standard interface. So the best we have is to use the ad-hoc interface required by our most likely class library candidate (Classpath), and simultaneously try to ignite interest in standardizing the aforementioned interface across many VMs. Well, that's one way. Another might be to start with the GNU Classpath interface and work out a standard interface based on a) further needs (i.e. what does 1.5 require?) b) spec/engineering considerations around java.lang (still to be hashed out) c) the experience of implementors like IBM/Sun/BEA that have done this too I'd be happy to offer a patch to GNU Classpath that supported such a beastie if we could ever come up with it. Obviously this is not going to gain us any leverage with existing proprietary VM s unless they also retrofit their library Well, if we pay attention to what they are doing, we may be able to pull this off. And why wouldn't they want to? They all are reinventing the wheel, a wheel that doesn't provide them with any advantages over their competitors - I can't see why they wouldn't want to standardize on a common approach and codebase :) - the only option in that case is to excise everything but java.lang.* from our bundling of Classpath and try to glue on the remaining portion of, say, Sun's library, or IBM's library; of course then those third party libraries must not cheat and use some of their own unpublished VM-specific interfaces, which they already do. Right - we need to figure out why they do it, what they need, and what we can use for our interface. The point is to stop the madness [vm] [vm-interface] [java.lang.* core (e.g. from Classpath)] [rest of the class library (e.g rest of Classpath, or some third party library)] I would not base any policy on support for developers breaking language rules. Yes you can cheat and use reflection to bypass visibility limitations (and I have even had to do this on some occasions to hack around some things), but you leave compatibility and portability at the door when you start doing such things. Where? in the VM? I think the VM can do whatever it wants, as long as the code executing on the VM is secure geir Aaron Tom Tromey wrote: Dan == Dan Lydick [EMAIL PROTECTED] writes: Dan That includes the language protection features like Geir's Dan example of package private, which are an interesting game of Dan navigating class file structures with reflection, etc. I've Dan never tried that, but Geir is right, I think it can be done if Dan you try hard enough. There are two cases. In the first case there is no security manager. All the code is trusted. In this case, yes, using reflection you can work around access protections. In practice I don't see how this matters, as the code is already trusted... if it does something weird, well, so what? It could also do weird things to your filesystem or anything else. In the second case, when there is a security manager, then, no, you cannot get around the access controls, even with reflection. Dan The underlying idea here is to make as few changes as possible to Dan as little of the java.*, especially java.lang.*, or other core Dan library packages in order to give the Harmony JVM runtime Dan environment the greatest flexibility for using libraries. Heck, Dan if it's done right, you might be able to use Sun's or IBM's Dan java.* library implementation! I see where you're coming from, but I don't see how this furthers the goal of Harmony, which as I understand it is to have a complete, compatible, open J2SE implementation. But then, I'm not
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Jun 3, 2005, at 4:34 PM, Tom Tromey wrote: Dan == Dan Lydick [EMAIL PROTECTED] writes: Dan That includes the language protection features like Geir's Dan example of package private, which are an interesting game of Dan navigating class file structures with reflection, etc. I've Dan never tried that, but Geir is right, I think it can be done if Dan you try hard enough. There are two cases. In the first case there is no security manager. All the code is trusted. In this case, yes, using reflection you can work around access protections. In practice I don't see how this matters, as the code is already trusted... if it does something weird, well, so what? It could also do weird things to your filesystem or anything else. In the second case, when there is a security manager, then, no, you cannot get around the access controls, even with reflection. Why would I have to get around the access controls? I'm the VM. I can do magic things, right? (I should be able to...) Dan The underlying idea here is to make as few changes as possible to Dan as little of the java.*, especially java.lang.*, or other core Dan library packages in order to give the Harmony JVM runtime Dan environment the greatest flexibility for using libraries. Heck, Dan if it's done right, you might be able to use Sun's or IBM's Dan java.* library implementation! I see where you're coming from, but I don't see how this furthers the goal of Harmony, which as I understand it is to have a complete, compatible, open J2SE implementation. Yes. I could believe that there are alternative class library implementations that a user might choose to use. With a clean, standard interface, the user should be able to just plug-n-play (modulo compatibility certification requirements) both the VM implementation and the class library implemetation. But then, I'm not really getting this part of the discussion. For instance, why does Harmony need a VM layer different from the one Classpath provides? I don't understand that. I'm not sure we don't, other than the bit about extending java.lang, which I just personally don't really like. I understand why it's done - to just take advantage of the language protection features - but I wonder if there are other ways that are equally as satisfactory for the intended purpose. geir Tom -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: ClassLib implementations WAS some ideas
I suspect the motivation behind the original post by Peter was more about formal modularization of the class library than general java package separation. I think that there has been some good work in this area in other places, such as larger scale J2EE systems via OSGi or -ish. Certainly modularization of the class library is in scope here (I think...) geir On Jun 3, 2005, at 2:22 PM, Tom Tromey wrote: Sven == Sven de Marothy [EMAIL PROTECTED] writes: Sven Depending on non-public parts of other API packages is to be Sven avoided as far as possible. And if there is Classpath-specific Sven behaviour in the public API, then that is likely a bug. (or the Sven absence of a Sun bug) One thing I've seen in user code is that it sometimes incorrectly depends on properties of a particular implementation. E.g., more than once I've run across Comparable implementations which are not symmetric, but which work because Sun's collections classes compare in a certain order. It isn't impossible that Classpath has some kind of subtle dependency like this somewhere. However, stuff like that, if it even exists, would just be a bug. i.e. how possible would it be to use say java.sql.* with another implementation of java.lang? Sven Why wouldn't it be possible? It'd be horrible coding if it were Sven any other way. Totally agreed. Most code in Classpath doesn't use native code, or VM code, or anything but the public API of classes in other packages. Tom -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
Geir Magnusson Jr. wrote: Dan That includes the language protection features like Geir's Dan example of package private, ... Sorry for not following up on some of my previous remarks on such things. But briefly, here's a trick that works, without any need for language-based module support (which would be much nicer, and will surely come for J2SE7, but isn't strictly needed in this case.) If you have functionality that should only be used across the packages inside the JDK library, and never by users, define it as (non-static) public method of a class (defined outside of java.* packages) that has no public constructors, but does have a factory method. That factory method includes the one bit of pure VM magic: It must only allow creation (or return a singleton) if the calling class is in bootclasspath, otherwise throwing some kind of exception. (The implementation of this check is just a specialization of other kinds of security checks JVMs must do.) Of course all JDK classes using such functionality should never pass around references to such classes. (This is one of the places where language support is needed). You'd like to define as few such classes as possible. It would be nice to standardize these across JVMs, but as I mentioned before, this is hard because JVM implementors want these internal APIs to remain flexible. This is probably the best way to handle all of those aspects of required Java APIs that cannot be described as bytecodes and/or require close Java-JVM coupling for the sake of efficiency. This scheme also applies to replacements for native in those cases where the implementation is not a call to native code, but is instead intrinsified, causing a JIT to produce custom instructions, as in the JikesRVM magic classes. I believe that the lack of such a facility might be one reason for some JVMs not yet supporting java.util.concurrent, which requires something like this to support compareAndSet instructions etc which must be intrinsified for performance to be acceptable. -Doug
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
From what I understood from this thread (and of course my understanding could be wrong), there is some contention over where to hide this Classlib-VM interface and implementation so that user code is least able to use/abuse it. One suggestion was to use existing package visibility modifiers and stash the classes in java.lang. Another was to take these classes and put them in a package other than java.lang to keep java.lang pure (at which point they presumably would have to be public). Another was to use classloader or VM magic (or perhaps some more sophisticated module publishing scheme) to hide the existence of these classes. In my humble opinion, I'm not overly concerned about spending a lot of effort to hide this code from application code, because it is already demonstrably wrong to use them (from user code), and with the proper measures one can easily circumvent the security manager and access hidden fields/methods anyway. Frankly, move-forward-with-Classpath-design-and-change-in-the-future-if-we-need-to sounds fine to me. Aaron Geir Magnusson Jr. wrote: On Jun 3, 2005, at 5:22 PM, Aaron Hamid wrote: I'm sort of confused by this discussion also. It seems to be about at what granularity we start introducing vm- specific interfaces/implementations. At some point the class library has to interface with the VM. We can do that at a very high level of course, by simply subsuming the entirety of java.lang.* and avoid this issue by simply making the java.lang package THE interface, Well, if GNU Classpath is any indication, you *can't* do that w/o extending java.lang, and therefore you aren't avoiding the issue - you are just burying it in java.lang. but we certainly don't want to rewrite and be responsible for all that code if we can avoid it. The only other alternative (at least that I have read) is some compromise whereby core classes rely on an even-more-core VM-specific API that will necessarily be different for each VM. Yes - and that would be opaque to the class library implementations... for example, the interface contract would specify what is needed for both sides (VM and class lib) and then it's up to the implementor of the VM to do the right thing wrt the interface w/ the full freedom to do it in any way they want, and similar for the class library implementation. As far as I can tell, there is no such standard interface. So the best we have is to use the ad-hoc interface required by our most likely class library candidate (Classpath), and simultaneously try to ignite interest in standardizing the aforementioned interface across many VMs. Well, that's one way. Another might be to start with the GNU Classpath interface and work out a standard interface based on a) further needs (i.e. what does 1.5 require?) b) spec/engineering considerations around java.lang (still to be hashed out) c) the experience of implementors like IBM/Sun/BEA that have done this too I'd be happy to offer a patch to GNU Classpath that supported such a beastie if we could ever come up with it. Obviously this is not going to gain us any leverage with existing proprietary VM s unless they also retrofit their library Well, if we pay attention to what they are doing, we may be able to pull this off. And why wouldn't they want to? They all are reinventing the wheel, a wheel that doesn't provide them with any advantages over their competitors - I can't see why they wouldn't want to standardize on a common approach and codebase :) - the only option in that case is to excise everything but java.lang.* from our bundling of Classpath and try to glue on the remaining portion of, say, Sun's library, or IBM's library; of course then those third party libraries must not cheat and use some of their own unpublished VM-specific interfaces, which they already do. Right - we need to figure out why they do it, what they need, and what we can use for our interface. The point is to stop the madness [vm] [vm-interface] [java.lang.* core (e.g. from Classpath)] [rest of the class library (e.g rest of Classpath, or some third party library)] I would not base any policy on support for developers breaking language rules. Yes you can cheat and use reflection to bypass visibility limitations (and I have even had to do this on some occasions to hack around some things), but you leave compatibility and portability at the door when you start doing such things. Where? in the VM? I think the VM can do whatever it wants, as long as the code executing on the VM is secure geir Aaron Tom Tromey wrote: Dan == Dan Lydick [EMAIL PROTECTED] writes: Dan That includes the language protection features like Geir's Dan example of package private, which are an interesting game of Dan navigating class file structures with reflection, etc. I've Dan never tried that, but Geir is right, I
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Sun, 2005-06-05 at 06:21 -0300, Geir Magnusson Jr. wrote: That's not fair. I told you that *I* think that extending java.lang is a *bad idea*. You many not agree, but that's not the same is NIH. That doesn't mean that java.lang.VMObject can't be move to another package, preserving the code in it's entirety. Yes, that's a change for VMs that were [lazily] depending on language semantics to protect those private package extensions, but that's life. Yes, well the impression I was getting was that the discussion here has drifted away from the pros and cons of the Classpath VM interface, and towards ideas of creating your own, or reimplementing java.lang, or even forking Classpath. Also, I don't know what the 1.4 and 1.5 Java API requirements on the VM are, but given that GNU Classpath isn't there yet, isn't it even plausible that may have something to add? Yes, 1.5 *will* add new requirements to the VM interface. But the Classpath VM interface isn't some immutable static thing either. All I'm saying here is: how about crossing that bridge when you get there, instead of deciding out of hand that it isn't good enough? /Sven
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Sun, 2005-06-05 at 06:25 -0300, Geir Magnusson Jr. wrote: On Jun 4, 2005, at 12:59 PM, Sven de Marothy wrote: AFAIK there are no other class libraries out there which you'll legally be able to distribute with Harmony. So why create flexibility when there aren't options? Are you kidding? There aren't options *now* (well, that's not really true, is it...), Could you elaborate on that? I don't know of any class library distributable under the Apache license. but that doesn't meant that options won't come around in the future. I think we're still in the very beginning of managed runtime environments and generalization w/o penalty is a Good Thing(tm). Reimplementing java.lang certainly is a penalty. Again, this is NOT a major issue. *If* or *when* these options become available, *that* will be the time to adress this. It is not such a major task as folks seem to think here to change the VM-classlib interface. Indeed it has been done already for VMs such as JikesRVM. Reimplementing java.lang is more work. And maybe we have more to learn in this area from other implementations and newer Java APIs. I don't like maybes. I like specific problems for which I can devise specific solutions. Maybe Java 1.6 will require VMs to be able to make breakfast; Should we start designing a VM-toaster interface, just in case? Why would you want to have a Free VM which can use non-free libraries? why not? Why restrict that freedom for users? 1) Because Sun hasn't documented their VM interface. 2) Because people who have Sun's class library already have Sun's VM. What would they want with Harmony for? 3) Because I thought the main idea was a complete VM under the Apache license. Not ASL+SCSL. /Sven
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
+1 to that! On Sun, 2005-06-05 at 11:53 -0400, Aaron wrote: From what I understood from this thread (and of course my understanding could be wrong), there is some contention over where to hide this Classlib-VM interface and implementation so that user code is least able to use/abuse it. One suggestion was to use existing package visibility modifiers and stash the classes in java.lang. Another was to take these classes and put them in a package other than java.lang to keep java.lang pure (at which point they presumably would have to be public). Another was to use classloader or VM magic (or perhaps some more sophisticated module publishing scheme) to hide the existence of these classes. In my humble opinion, I'm not overly concerned about spending a lot of effort to hide this code from application code, because it is already demonstrably wrong to use them (from user code), and with the proper measures one can easily circumvent the security manager and access hidden fields/methods anyway. Frankly, move-forward-with-Classpath-design-and-change-in-the-future-if-we-need-to sounds fine to me. Aaron
Re: JVM Spec version 2 and .class file versioning
Dan == Dan Lydick [EMAIL PROTECTED] writes: Dan What gives? Where can I find correct descriptions of Dan the latest .class file definitions for JDK 5? Here's one source I found via google. Not sure if it is the most up to date. http://java.sun.com/docs/books/vmspec/2nd-edition/UpdatedClassFileFormat.pdf AIUI, the major version in 1.4 was 48 and the major version in 1.5 is 49. There are differences between 1.3, 1.4, and 1.5. As I remember, 1.4 introduce a little difference in verification. 1.5 adds a bunch of new things, including an extension to the semantics of 'ldc'. Tom
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
Geir == Geir Magnusson [EMAIL PROTECTED] writes: Geir That's not fair. I told you that *I* think that extending java.lang Geir is a *bad idea*. I still don't understand why you think this. Could you explain what problem might arise? Geir Also, I don't know what the 1.4 and 1.5 Java API requirements on the Geir VM are, but given that GNU Classpath isn't there yet, isn't it even Geir plausible that may have something to add? Yeah, the VM interface has to be extended in a few ways for 1.5. Also from time to time the VM interface has to be extended for new VMs, which have different needs. This is one reason that the interface isn't yet stable, even for 1.4. Tom
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Jun 5, 2005, at 1:30 PM, Sven de Marothy wrote: On Sun, 2005-06-05 at 06:21 -0300, Geir Magnusson Jr. wrote: That's not fair. I told you that *I* think that extending java.lang is a *bad idea*. You many not agree, but that's not the same is NIH. That doesn't mean that java.lang.VMObject can't be move to another package, preserving the code in it's entirety. Yes, that's a change for VMs that were [lazily] depending on language semantics to protect those private package extensions, but that's life. Yes, well the impression I was getting was that the discussion here has drifted away from the pros and cons of the Classpath VM interface, and towards ideas of creating your own, or reimplementing java.lang, or even forking Classpath. Fair enough. But I'd really like to rope this back to the pros and cons of the GNU Classpath VM interface. Also, I don't know what the 1.4 and 1.5 Java API requirements on the VM are, but given that GNU Classpath isn't there yet, isn't it even plausible that may have something to add? Yes, 1.5 *will* add new requirements to the VM interface. But the Classpath VM interface isn't some immutable static thing either. All I'm saying here is: how about crossing that bridge when you get there, instead of deciding out of hand that it isn't good enough? Guess which version of J2SE we want to do... geir /Sven -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
Geir == Geir Magnusson [EMAIL PROTECTED] writes: Dan I've Dan never tried that, but Geir is right, I think it can be done if Dan you try hard enough. In the second case, when there is a security manager, then, no, you cannot get around the access controls, even with reflection. Geir Why would I have to get around the access controls? I'm the VM. I Geir can do magic things, right? (I should be able to...) I thought we were talking about malicious user code. Tom
RE: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
Geir Magnusson Jr. wrote: Geir Magnusson Jr. wrote: True, except I really hate extending java.lang. :) What do you mean by extending java.lang? GNU Classpath does not extend java.lang (for any reasonable definition of extend). Having package private classes is not extending. Why do you say that? Suppose for J2SE 6, the EG adds the public class java.lang.VMObject. then what? Then we rename it. A new Java version will involve changes to the VM interface anyway, so this makes no difference. Regards, Jeroen
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
Geir == Geir Magnusson [EMAIL PROTECTED] writes: What do you mean by extending java.lang? GNU Classpath does not extend java.lang (for any reasonable definition of extend). Having package private classes is not extending. Geir Why do you say that? Suppose for J2SE 6, the EG adds the public Geir class java.lang.VMObject. then what? I already explained how this can't cause a user-visible problem. Maybe you could explain what problem you see arising. As I see it there are a few cases here: * Someone takes code that uses this 1.6 feature and: 1. Compiles it against our library. - this fails as our VMObject is package-private - This is no different from using any other unimplemented class 2. Compiles it against 1.6 and runs it against our library - this fails as our VMObject is package-private - This is no different from using any other unimplemented class * Someone takes a part of our java.lang, which uses our VMObject, and runs them on a 1.6 VM - Yes, this fails. But this is not supported either upstream (there is no supported way to override parts of the class library, other than those parts mentioned in the endorsed dirs spec) or by us (since it is a silly thing to do) * Note that user code can't use our VMObject as it is not user-visible Renaming our VMObject in our next release causes no problems for any code. The VMs have to change, but then the VMs have to change with each major release anyway, since ordinarily new features are added which require new VM glue. Tom
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Jun 5, 2005, at 1:45 PM, Sven de Marothy wrote: On Sun, 2005-06-05 at 06:25 -0300, Geir Magnusson Jr. wrote: On Jun 4, 2005, at 12:59 PM, Sven de Marothy wrote: AFAIK there are no other class libraries out there which you'll legally be able to distribute with Harmony. So why create flexibility when there aren't options? Are you kidding? There aren't options *now* (well, that's not really true, is it...), Could you elaborate on that? I don't know of any class library distributable under the Apache license. There are other licenses. Remember, we aren't only concerned about ourselves, but what a downstream user of our stuff will want to do. We tend to try to protect their freedoms as well. (See Ulcer, Geir's, re J2EE TCK license) but that doesn't meant that options won't come around in the future. I think we're still in the very beginning of managed runtime environments and generalization w/o penalty is a Good Thing(tm). Reimplementing java.lang certainly is a penalty. I don't understand - I might have misstated something. Why do you think I want to re-implement java.lang? Any JVM that uses GNU Classpath has to implement java.lang parts, right? All I'm suggesting that we move the stuff that's not standard java.lang as defined in a spec somewhere off to another package name. Again, this is NOT a major issue. *If* or *when* these options become available, *that* will be the time to adress this. It is not such a major task as folks seem to think here to change the VM-classlib interface. Indeed it has been done already for VMs such as JikesRVM. Why not do it now so we don't have to fix it later, since to do J2SE 5 we *are* going to have to modify it... Reimplementing java.lang is more work. See above - I think there is a miscommunication here And maybe we have more to learn in this area from other implementations and newer Java APIs. I don't like maybes. I like specific problems for which I can devise specific solutions. Me too, and I'm hoping someone who has done this will point out some specific problems they needed to solve. Maybe Java 1.6 will require VMs to be able to make breakfast; Should we start designing a VM-toaster interface, just in case? As long as you don't put it in java.lang, I'm all for it... :) But before we go leaping off to 1.6, how about 1.5? Why would you want to have a Free VM which can use non-free libraries? why not? Why restrict that freedom for users? 1) Because Sun hasn't documented their VM interface. We don't care, do we? We can do our own. 2) Because people who have Sun's class library already have Sun's VM. What would they want with Harmony for? Ya never know :) 3) Because I thought the main idea was a complete VM under the Apache license. Not ASL+SCSL. Remember the modularity goal - we want people to be able to take this stuff and plug-n-play with whatever they want. If for whatever reason they wanted to plug in Sun's class library, why would we want to prevent that? geir -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Jun 5, 2005, at 2:05 PM, Tom Tromey wrote: Geir == Geir Magnusson [EMAIL PROTECTED] writes: Geir That's not fair. I told you that *I* think that extending java.lang Geir is a *bad idea*. I still don't understand why you think this. Could you explain what problem might arise? Sun adds java.lang.VMObject as a public class (ok, farfetched) or adds it as a private class (not so farfetched). Geir Also, I don't know what the 1.4 and 1.5 Java API requirements on the Geir VM are, but given that GNU Classpath isn't there yet, isn't it even Geir plausible that may have something to add? Yeah, the VM interface has to be extended in a few ways for 1.5. Also from time to time the VM interface has to be extended for new VMs, which have different needs. This is one reason that the interface isn't yet stable, even for 1.4. It would be interesting to know why and how it was extended - there's something to learn there. I'm also interested in getting some input from the commercial VM developers. geir Tom -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Jun 5, 2005, at 2:12 PM, Jeroen Frijters wrote: Geir Magnusson Jr. wrote: Geir Magnusson Jr. wrote: True, except I really hate extending java.lang. :) What do you mean by extending java.lang? GNU Classpath does not extend java.lang (for any reasonable definition of extend). Having package private classes is not extending. Why do you say that? Suppose for J2SE 6, the EG adds the public class java.lang.VMObject. then what? Then we rename it. A new Java version will involve changes to the VM interface anyway, so this makes no difference. I assume you'd keep it in java.lang? Isn't that just sliding the deck chairs around? geir Regards, Jeroen -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Jun 5, 2005, at 2:06 PM, Tom Tromey wrote: Geir == Geir Magnusson [EMAIL PROTECTED] writes: Dan I've Dan never tried that, but Geir is right, I think it can be done if Dan you try hard enough. In the second case, when there is a security manager, then, no, you cannot get around the access controls, even with reflection. Geir Why would I have to get around the access controls? I'm the VM. I Geir can do magic things, right? (I should be able to...) I thought we were talking about malicious user code. Ah - sorry. I was tired and on a plane home from Brazil. I though we were talking about how the code running in the context of the VM (as opposed to user code) could get ahold of the necessary classes (and protect them from malicious user code) geir Tom -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
RE: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
Geir Magnusson Jr. wrote: On Jun 5, 2005, at 2:12 PM, Jeroen Frijters wrote: Geir Magnusson Jr. wrote: Geir Magnusson Jr. wrote: True, except I really hate extending java.lang. :) What do you mean by extending java.lang? GNU Classpath does not extend java.lang (for any reasonable definition of extend). Having package private classes is not extending. Why do you say that? Suppose for J2SE 6, the EG adds the public class java.lang.VMObject. then what? Then we rename it. A new Java version will involve changes to the VM interface anyway, so this makes no difference. I assume you'd keep it in java.lang? Isn't that just sliding the deck chairs around? No, it's a trivial cost. Your proposed solution is in reality much more expensive. You're missing the fact that moving these classes into another packages creates another problem that is much worse. Namely that you often need to access private state in the public classes, you can do that by living in the same package, that's why the VM* classes live in the same package as their public counterpart. Regards, Jeroen
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
Geir == Geir Magnusson [EMAIL PROTECTED] writes: I still don't understand why you think this. Could you explain what problem might arise? Geir Sun adds java.lang.VMObject as a public class (ok, farfetched) or Geir adds it as a private class (not so farfetched). Ok, this is the scenario. This I understand. But what actual problem does this cause? Tom
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
I actually had not considered this issue which would seem to warrant these classes living in the same package. But can not an equivalent solution be constructed such that the implementations of these public classes can hand the VM* classes references to internal structures (and vice versa) though well defined interfaces, instead of relying on visibility modifiers to allow the VM* objects to access private state of java.lang classes, thereby allowing the VM* objects to live in a separate package? Or am I misunderstanding your statement (or maybe that is just too cumbersome)? Aaron Jeroen Frijters wrote: You're missing the fact that moving these classes into another packages creates another problem that is much worse. Namely that you often need to access private state in the public classes, you can do that by living in the same package, that's why the VM* classes live in the same package as their public counterpart. Regards, Jeroen
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
Geir == Geir Magnusson [EMAIL PROTECTED] writes: Yes, 1.5 *will* add new requirements to the VM interface. But the Classpath VM interface isn't some immutable static thing either. All I'm saying here is: how about crossing that bridge when you get there, instead of deciding out of hand that it isn't good enough? Geir Guess which version of J2SE we want to do... So far nobody has systematically looked at all the VM layer changes needed for 1.5. However, I have looked at a few. The ones I've looked at are mostly related to Class or reflection; we need to add some more methods to VMClass and friends to account for the new data that is available. We also needed VMClass.cast(), a change to VMSystem for the new getenv, and a change to VMProcess for ProcessBuilder. As far as I know we won't need any major overhauls, just additions here and there. But if someone wanted to go through all the new bits and check this, that would be helpful. Better yet, actually go write the new bits, there is plenty of 1.5 stuff still to be done. Tom
Re: Summer of Code, something for Harmony?
El dom, 05-06-2005 a las 02:00 -0500, Dan Lydick escribi: [Original Message] From: Santiago Gala [EMAIL PROTECTED] To: harmony-dev@incubator.apache.org Date: 6/4/05 1:22:40 PM Subject: Re: Summer of Code, something for Harmony? ...snip... El jue, 02-06-2005 a las 02:11 +0200, Leo Simons escribi=C3=B3: JikesRVM has no current class verifier, neither jamvm or sableVM. As far as I know, the spec mandates to have class verification (I just read it in my JVM Spec (a bit old). Which version do you have. (I just posted a question directly related to versioning of the JVM spec, title is JVM Spec version 2 and .class file versioning.) I have the paper version of The Java Virtual Machine Specification. Second Edition. I bought it in 1999, together with the IBM boot on java security which included 1.2 features. I found interesting the errata and addenda PDFs someone pointed to. Long time since I looked last into the internals of java. I would like to have a Fifth Edition rewritten to take into account java 1.5, but marking additions/corrections since 1.2 by version... (is it out there?) Regards Santiago -- VP and Chair, Apache Portals (http://portals.apache.org) Apache Software Foundation Dan Lydick -- VP and Chair, Apache Portals (http://portals.apache.org) Apache Software Foundation signature.asc Description: This is a digitally signed message part
Re: [arch] How much of java.* and friends does Harmony need towrite. Was: VM/Classlibrary interface
El dom, 05-06-2005 a las 01:56 -0500, Dan Lydick escribi: (...) I suppose I need to do some more homework then. If GNU Classpath is used without modification for numerous VM's, then great! Ultimately, what I am A few classes need to be modified: - jamvm (from README): A number of classes are reference classes which must be modified for a particular VM. These are provided and built along with JamVM (see INSTALL). JamVM 1.3.0 works with Classpath version 0.14 (the latest development snapshot). It should also work with the latest version direct from CVS (but check the Classpath mailing lists for possible patches). Later snapshots of Classpath may also work, if no modifications have been made to the VM interface. - jikesRVM fetches classpath from cvs, and allows you to put your own copy instead, to build the boot image. In addition, it implements or modifies, glancing at cvs code, gnu.classpath.(VMStackWalker.java| VMSystemProperties.java) and a bunch of classes (VM) in java.(lang|io| net|util).VM* (including also Object, Thread, Throwable .ref.*, some .reflect. classes) - sableVM (1.11.3) bundles a version of classpath with it, numbered after the sableVM release. I'm not sure how modified it is WRT CVS classpath, but the claim is SableVM Classpath is based on GNU Classpath 0.08 + further CVS changes (INSTALL-DEVEL) - not sure about other VMs This is inconvenient for VM writers, as they must bundle or track the development of those classes in both libraries. I think that this is why there was a suggestion to isolate all references in a separate deliverable (be it package, jar, bundle, whatever), so that people working in different VMs can have independent versioning and API changes than the library as a whole. This will become specially handy as classpath grows, because the VM* part will be fairly stable, but changes as implementation gets better or new APIs (JSR) get approved will be big. Regards Santiago -- Santiago Gala High Sierra Technology, S.L. http://memojo.com/blojsom/blog/sgala/ signature.asc Description: This is a digitally signed message part
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Sun, 5 Jun 2005, Geir Magnusson Jr. wrote: Also from time to time the VM interface has to be extended for new VMs, which have different needs. This is one reason that the interface isn't yet stable, even for 1.4. It would be interesting to know why and how it was extended - there's something to learn there. I'm also interested in getting some input from the commercial VM developers. The new features in 1.5 varargs, enums, annotations, and generics required additions to the class file format. More changes appear to be introduced to improve debugging (type infos about local variables). Tom Tromey has already found a link to an updated class file description: http://java.sun.com/docs/books/vmspec/2nd-edition/UpdatedClassFileFormat.pdf Here is a short and probably incomplete overview over the changes: varargs:ACC_VARARGS, ACC_BRIDGE (?) enums: ACC_ENUM annotations:ACC_ANNOTATION New attributes: EnclosingMethod, Signature (for generics), LocalVariableTypeTable, RuntimeVisibleAnnotations, RuntimeInvisibleAnnotations, RumtimeVisibleParameterAnnotations, RumtimeInvisibleParameterAnnotations, Uli
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Sun, Jun 05, 2005 at 06:20:37AM -0300, Geir Magnusson Jr. wrote: On Jun 3, 2005, at 6:39 PM, Dalibor Topic wrote: That's a rather weird thing to standardise, being the internals of a binding between the library and the VM, and necessarily something that is in flux, as VMs and class libraries change internally. And they do change quite a bit over time ;) I assume they only change as the Java API changes? I.e. the interface needed for 1.4 could be different than the interface needed for 1.5 etc? (and similarly, if the interface is different, it implies a required change on both sides of it ?) They change according to everyone'S needs. That may be chanes in the API specs that require native or VM specific support, but also may be cleanups, optimizations, bug fixes, and so on. If that's so, then I think it is reasonable to standardize - you'd just have the standard interface for a given Java API version... So far, things have continued to evolved quickly to accomodate the needs of all different VMs in classpath, and quite effectively. So, as lng as everything continues to be in flux, i doubt it will be something to standardise on right now. the changes are proposed on the classpath mailing list, and discussed there. cheers, dalibor topic geir -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface
On Sun, Jun 05, 2005 at 02:20:53PM -0300, Geir Magnusson Jr. wrote: On Jun 5, 2005, at 1:45 PM, Sven de Marothy wrote: On Sun, 2005-06-05 at 06:25 -0300, Geir Magnusson Jr. wrote: On Jun 4, 2005, at 12:59 PM, Sven de Marothy wrote: AFAIK there are no other class libraries out there which you'll legally be able to distribute with Harmony. So why create flexibility when there aren't options? Are you kidding? There aren't options *now* (well, that's not really true, is it...), Could you elaborate on that? I don't know of any class library distributable under the Apache license. There are other licenses. Remember, we aren't only concerned about ourselves, but what a downstream user of our stuff will want to do. We tend to try to protect their freedoms as well. (See Ulcer, Geir's, re J2EE TCK license) Could you elaborate on those things? You sound as if there is some option for downstream users on your mind, but you are not coming out with it, and it is not very clear what it is from guessing what you might be thinking of ;) but that doesn't meant that options won't come around in the future. I think we're still in the very beginning of managed runtime environments and generalization w/o penalty is a Good Thing(tm). Reimplementing java.lang certainly is a penalty. I don't understand - I might have misstated something. Why do you think I want to re-implement java.lang? Any JVM that uses GNU Classpath has to implement java.lang parts, right? All I'm suggesting that we move the stuff that's not standard java.lang as defined in a spec somewhere off to another package name. I beliee the confusion exists because someone else suggested reimplementing all of java.lang with a rather odd justification, to be able to mix in SCSLd code, that is not practicable nor legally possible to distribute ;) Your argumentation jumped right into the middle of that and that may have lead people to assume wrong things. Maybe you should simply start a new thread, present a use case, show what breaks, and we can analyze it. Hunting things down in this long htread seems to be very confusing for everyone. Again, this is NOT a major issue. *If* or *when* these options become available, *that* will be the time to adress this. It is not such a major task as folks seem to think here to change the VM-classlib interface. Indeed it has been done already for VMs such as JikesRVM. Why not do it now so we don't have to fix it later, since to do J2SE 5 we *are* going to have to modify it... Given that no such options exist, it is not possible to make any ammendments for them right now, right, as noone knws what ammendments they would need? ;) What are the options on your mind? Could you give us some nice scenarios, name some names, to see what motivates the discussion and how we can fix the the problems in the VM interface for those users and/or clear up any misconceptions about it. cheers, dalbor topic Reimplementing java.lang is more work. See above - I think there is a miscommunication here And maybe we have more to learn in this area from other implementations and newer Java APIs. I don't like maybes. I like specific problems for which I can devise specific solutions. Me too, and I'm hoping someone who has done this will point out some specific problems they needed to solve. Maybe Java 1.6 will require VMs to be able to make breakfast; Should we start designing a VM-toaster interface, just in case? As long as you don't put it in java.lang, I'm all for it... :) But before we go leaping off to 1.6, how about 1.5? Why would you want to have a Free VM which can use non-free libraries? why not? Why restrict that freedom for users? 1) Because Sun hasn't documented their VM interface. We don't care, do we? We can do our own. 2) Because people who have Sun's class library already have Sun's VM. What would they want with Harmony for? Ya never know :) 3) Because I thought the main idea was a complete VM under the Apache license. Not ASL+SCSL. Remember the modularity goal - we want people to be able to take this stuff and plug-n-play with whatever they want. If for whatever reason they wanted to plug in Sun's class library, why would we want to prevent that? geir -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
[arch] VM Interface
Hi, It seems like there is a little bit of heat being generated by this topic due to confusion. While Geir has not actually stated this anywhere I assume that the reason that he is advocating for a VM interface that is independent of GNU Classpath is because he has plans to interoperate with other class libraries. I assume that if the Harmony JVM gets half as good as is hoped there will be companys who want to adopt the JVM but continue to use Suns class library so that differences in libraries don't hurt their customers. Consider IBM - There are a few people here (both active and lurkers) that are IBMers. They have publicly showed support for an open source JVM on numerous occasions and I am sure they would benefit considerably (as would Harmony) if this project was to get to that stage. However I suspect that it is likely that they want to stick with a derivative of Suns rt.jar for the moment. The reason being that their customers do not want to be exposed to differences between rt.jar and GNU Classpath. Given that IBM has already re-written large chunks of the JVM I suspect that over time they may move piecemeal to an OSS class library - at a pace at which they can verify it matches Suns behaviour. Another possibility would be the people from Brazil who are starting their own JVM and I would not be surprised if at some point someone wants to reimplement the class library using with a ASL/MIT or other license with fewer restrictions. I could be wrong but I guess the idea is to keep the options open and encourage collaboration with both big buisness and other OSS projects. --- Cheers, Peter Donald ** | You can't wake a person who is pretending | | to be asleep. -Navajo Proverb. | **
Re: [arch] VM Interface
One potential use is for companies (and individuals) to work around particular performance limitations and bugs in the Sun VM while keeping the libraries they know inside and out. I imagine that could become common if Harmony ends up being as modular as we all hope. I am curious as to how much of the standard libraries would be rendered non-functional without the VM specific classes from Sun, however. Craig Blake On Jun 5, 2005, at 5:32 PM, Peter Donald wrote: Hi, It seems like there is a little bit of heat being generated by this topic due to confusion. While Geir has not actually stated this anywhere I assume that the reason that he is advocating for a VM interface that is independent of GNU Classpath is because he has plans to interoperate with other class libraries. I assume that if the Harmony JVM gets half as good as is hoped there will be companys who want to adopt the JVM but continue to use Suns class library so that differences in libraries don't hurt their customers. Consider IBM - There are a few people here (both active and lurkers) that are IBMers. They have publicly showed support for an open source JVM on numerous occasions and I am sure they would benefit considerably (as would Harmony) if this project was to get to that stage. However I suspect that it is likely that they want to stick with a derivative of Suns rt.jar for the moment. The reason being that their customers do not want to be exposed to differences between rt.jar and GNU Classpath. Given that IBM has already re-written large chunks of the JVM I suspect that over time they may move piecemeal to an OSS class library - at a pace at which they can verify it matches Suns behaviour. Another possibility would be the people from Brazil who are starting their own JVM and I would not be surprised if at some point someone wants to reimplement the class library using with a ASL/MIT or other license with fewer restrictions. I could be wrong but I guess the idea is to keep the options open and encourage collaboration with both big buisness and other OSS projects. --- Cheers, Peter Donald ** | You can't wake a person who is pretending | | to be asleep. -Navajo Proverb. | **
Re: [arch] VM Interface
On Jun 5, 2005, at 9:32 PM, Peter Donald wrote: Hi, It seems like there is a little bit of heat being generated by this topic due to confusion. While Geir has not actually stated this anywhere I assume that the reason that he is advocating for a VM interface that is independent of GNU Classpath is because he has plans to interoperate with other class libraries. I thought that was a base assumption of the project - that a modular architecture provides ways for different implementations to interoperate. Goal #2 from the proposal : 2) create a community-developed modular runtime (VM and class library) architecture to allow independent implementations to share runtime components, and allow independent innovation in runtime components I assume that if the Harmony JVM gets half as good as is hoped there will be companys who want to adopt the JVM but continue to use Suns class library so that differences in libraries don't hurt their customers. Or write their own class libraries for whatever reason. Consider IBM - There are a few people here (both active and lurkers) that are IBMers. I'm an IBM-er but can't and won't comment on IBMs position on this. (I work in a completely different area of the company. It's a big company...) They have publicly showed support for an open source JVM on numerous occasions and I am sure they would benefit considerably (as would Harmony) if this project was to get to that stage. However I suspect that it is likely that they want to stick with a derivative of Suns rt.jar for the moment. The reason being that their customers do not want to be exposed to differences between rt.jar and GNU Classpath. Given that IBM has already re-written large chunks of the JVM I suspect that over time they may move piecemeal to an OSS class library - at a pace at which they can verify it matches Suns behaviour. Another possibility would be the people from Brazil who are starting their own JVM and I would not be surprised if at some point someone wants to reimplement the class library using with a ASL/MIT or other license with fewer restrictions. I could be wrong but I guess the idea is to keep the options open and encourage collaboration with both big buisness and other OSS projects. Yes - exactly. geir --- Cheers, Peter Donald ** | You can't wake a person who is pretending | | to be asleep. -Navajo Proverb. | ** -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] VM Interface
On Jun 5, 2005, at 10:06 PM, Craig Blake wrote: One potential use is for companies (and individuals) to work around particular performance limitations and bugs in the Sun VM while keeping the libraries they know inside and out. I imagine that could become common if Harmony ends up being as modular as we all hope. I am curious as to how much of the standard libraries would be rendered non-functional without the VM specific classes from Sun, however. Agreed. But there is a reasonable argument to be made that it's in Sun's interest too to see a common interface/bridge/foo between the VM and class library, so lets not count them out of this. This is a long project, and it's still early :) geir Craig Blake On Jun 5, 2005, at 5:32 PM, Peter Donald wrote: Hi, It seems like there is a little bit of heat being generated by this topic due to confusion. While Geir has not actually stated this anywhere I assume that the reason that he is advocating for a VM interface that is independent of GNU Classpath is because he has plans to interoperate with other class libraries. I assume that if the Harmony JVM gets half as good as is hoped there will be companys who want to adopt the JVM but continue to use Suns class library so that differences in libraries don't hurt their customers. Consider IBM - There are a few people here (both active and lurkers) that are IBMers. They have publicly showed support for an open source JVM on numerous occasions and I am sure they would benefit considerably (as would Harmony) if this project was to get to that stage. However I suspect that it is likely that they want to stick with a derivative of Suns rt.jar for the moment. The reason being that their customers do not want to be exposed to differences between rt.jar and GNU Classpath. Given that IBM has already re-written large chunks of the JVM I suspect that over time they may move piecemeal to an OSS class library - at a pace at which they can verify it matches Suns behaviour. Another possibility would be the people from Brazil who are starting their own JVM and I would not be surprised if at some point someone wants to reimplement the class library using with a ASL/MIT or other license with fewer restrictions. I could be wrong but I guess the idea is to keep the options open and encourage collaboration with both big buisness and other OSS projects. --- Cheers, Peter Donald ** | You can't wake a person who is pretending | | to be asleep. -Navajo Proverb. | ** -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Pluggable bytecode verifiers (was RE: Summer of Code, something for Harmony?)
Bytecode verifiers appear to be one thing where the Harmony project could contribute some work in standardising interfaces. At the moment I know of two open source verifiers written in Java: BCEL (http://jakarta.apache.org/bcel/index.html, Apache Licence) and ASM (http://asm.objectweb.org/, BSD Licence) I believe that GCJ also has a verifier which it is (planning on?) sharing with Kaffe, but I haven't looked at that. The BCEL verifier can be used from the Verifier class http://jakarta.apache.org/bcel/apidocs/org/apache/bcel/verifier/Verifier html The ASM verifier can be used from CheckClassAdapter: http://asm.objectweb.org/current/doc/javadoc/user/org/objectweb/asm/util /CheckClassAdapter.html From a short investigation, both verifiers have their advantages: for instance BCEL provides more complete verification, while ASM is faster and provides some JDK1.5 features. A more complete comparison can be found at http://mail-archives.apache.org/mod_mbox/jakarta-bcel-dev/200503.mbox/% [EMAIL PROTECTED] 3E and http://www.mail-archive.com/bcel-dev@jakarta.apache.org/msg00631.html It would appear to be a relatively simple task to create an interface that wrapped either of these verifiers. Presumably it would be no more work to integrate that common interface than it would be to integrate the native BCEL verifier. [snip] I posted a proposal in the wiki http://wiki.apache.org/general/SummerOfCode2005 It is about integrating the BCEL class verifier with one or more OS JVM (written in C or Java). JikesRVM has no current class verifier, neither jamvm or sableVM. As far as I know, the spec mandates to have class verification (I just read it in my JVM Spec (a bit old). I have been testing how feasible it is, and it looks the right size for someone interested in understanding this world, and something doable in a summer. [snip] IMPORTANT: This e-mail, including any attachments, may contain private or confidential information. If you think you may not be the intended recipient, or if you have received this e-mail in error, please contact the sender immediately and delete all copies of this e-mail. If you are not the intended recipient, you must not reproduce any part of this e-mail or disclose its contents to any other party. This email represents the views of the individual sender, which do not necessarily reflect those of education.au limited except where the sender expressly states otherwise. It is your responsibility to scan this email and any files transmitted with it for viruses or any other defects. education.au limited will not be liable for any loss, damage or consequence caused directly or indirectly by this email.
Re: Other interesting papers and research
Hi Steve and all, | The approach of using C Compiler generated code rather than writing a | full compiler appeals to me: | http://www.csc.uvic.ca/~csc586a/papers/ertlgregg04.pdf From: Steve Blackburn [EMAIL PROTECTED] Date: Tue, 24 May 2005 21:08:05 +1000 They automatically build themselves simple JIT backends (by extracting fragments produced by the ahead of time compiler). This sounds like a great way to achieve portability while achiving better performance than a conventional interpreter. I guess it's a bit better or just comparable with a good interpreter. They say it is a lot better: speedups of up to 1.87 over the fastest previous interpreter based technique, and performance comparable to simple native code compilers. Their technique may be reasonable in cases, but it is better to be aware of: - On an Athlon processor, speedup (gforth-native over gforth-super) is less than those on PowerPC. It is between 1.06 and 1.49. - The machine code concatinating technique consumes much memory. In my experience, generated machine code is about 10 times larger than the original instructions in Java bytecode. In the paper, the authors have not mentioned memory consumption of the technique. We cannot guess how much it is precisely, but it is possible to be a big drawback. Yes, we can say the same for the approach taking a baseline compiler instead of an interpreter (like Jikes RVM). Memory consumption of the baseline compiler of Jike RVM is very interesting. And the next one does not reduce the value of the technique, but better to know: - The compared interpreter (gforth-super) could be improved by other techniques including stack caching. This means that their machine code concatinating technique may benefit from those remaining techniques. In 1998, I have written such a JIT compiler concatinate code fragments generated by GCC for each JVM instruction. Very interesting! Unfortunately, the JIT was slightly slower than an interpreter in Sun's Classic VM. The interpreter was written in x86 assembly language and implements dynamic stack caching with 2 registers and 3 states. It performs much better than the previous interpreter written in C. Then I rewrote the JIT. In reality, my old JIT compiler in C cannot be strictly compared to the JDK interpreter in assembly language. I try clarifying the situation. inst. executionstack handling (1)ErtlGregg's gforth-super direct threading 2 TOS in a register 2 (2)ErtlGregg's gforth-native concat'd machine code 1 TOS in a register 2 (3)JDK interpreter in Cswitch threading 3 memory3 (4)JDK interpreter in asm direct threading 2 stack caching 1 (5)Shudo's first JIT concat'd machine code 1 memory3 (6)Shudo's lightweight JIT concat'd macihne code 1 stack caching 1 Gforth-super (1) is the ErtlGregg's fastest interpreter compared to the machine code concatinating technique (2). (3) is the only interpreter which an ancient JDK 1.0.2 had. (4) is an interpreter of JDK 1.1 written in assembly language. (5) is my first JIT compiler which concatinates machine code for each VM instruction. (6) is shuJIT, the JIT compiler re-written after (5). 1-3 mean the order of expected performance. Machine code concatination proposed in the ErtlGregg's paper is marked with 1 because it is the best in those in this table for VM instruction execution. (4) and (6) uses 2 machine registers for stack caching and ErtlGregg's implementations uses one register to cache the top of stack (TOS). ErtlGregg's paper says that gforth-native (2) provided speedups up to 1.87 over gforth-super (1). I wrote in a previous posting that (5) was slower than (4), but (5) was inferior to (4) in stack handling. It is possible to be the reason of the slowness and then I cannot say that the machine code concatinating technique is not necessarily faster than a good interpreter like (4). The followings are other facts I experienced: - Shudo's lightweight JIT (6) generates about 10 times larger native code compared with the original Java bytecode instructions. This is possible to be a drawback of an approach taking a baseline compiler instead of an interpreter. Does the ErtlGregg's machine concatinating technique suffer it? How about a baseline compiler of Jikes RVM? - An interpreter of JDK 1.1 written in assembly (4) executed a numerical benchmark, Linpack about twice as fast as the previous interpreter in C (3). A lesson here is that an interpreter is often regarded as just slow but a faster one and a slower one are very different in performance. - Register utilization is still important even for an interpreter. Interpreters (1) and (4), and lightweight JIT compilers (2) and (6) utilizes one or two machine registers to cache values around TOS. It is also important to map VM registers (e.g. PC) onto machine registers.
