Re: [arch] VMCore / Component Model
I have now added the prototype implementation to JIRA: http://issues.apache.org/jira/browse/HARMONY-5 Cheers, David. On Mon, 2005-09-19 at 19:46 +0200, David Tanzer wrote: Today I have added some additional Information to the Wiki page. We should also consider using C++ and abstract classes to maintain our component model. While this would make inter-component communication slightly slower it would be easier to maintain. We should also think about using an existing component model like OSGi. The model I posted provides pretty fast communication between components without sacrificing too much flexibility, but it is maybe not as easy to maintain as a clean, object-oriented implementation (i.e. C++). We could discuss how important these aspects are to us, i.e. how much runtime efficiency we are willing to sacrifice for maintainability and flexibility and vice-versa. Regards, David. On Fri, 2005-09-16 at 21:47 +0200, David Tanzer wrote: Ok, it took a little bit longer than I first expected, but now my proof-of-concept implementation of the component model I described is available in the wiki: http://wiki.apache.org/harmony/ComponentModelFunctionPointers I have also linked it from the harmony architecture page. It contains a mechanism for loading components and a basic versioning and dependency management mechanism. The test case loads two components, where one depends on the other. I'll add some more explanations to the wiki page when I have more time, hopefully at the weekend. I have made several assumptions about the directory structure, the coding conventions and the documentation conventions, we should maybe discuss this in a different thread. Regards, David. On Tue, 2005-09-13 at 17:27 +0100, Tim Ellison wrote: David Tanzer wrote: Since we already started to define some component interfaces I think we also should start thinking about a component model which loads / connects such components. Maybe there are also some existing solutions we might want to look at (I must confess I didn't really search yet). Agreed, plus managing the API itself to ensure forwards/backwards version compatibility. I guess a requirement for such a component manager would be that it can load and connect components at runtime and that the specific implementations which are loaded can be configured. It might also be good if the same component implementations can be linked at compile time (i.e. statically) which could have benefits on embedded platforms, but I'm not sure if we really need this. I'm assuming that you are speculating on component management beyond the platform support (i.e. DLL-hell). The java world is already on this path with the OSGi framework (e.g. Felix). It will require a non-trivial solution to ensure that the runtime flexibility does not incur an unacceptable runtime cost. Another requirement would be that the components can be written in different programming languages (i.e. C, C++, Java, ...). It isn't really a problem to solve this for C and C++, but can we also easily support other programming languages? A simple way to implement such a component model in C would be an approach similar to the one Tim Ellison described in [1] where he explains the structure of the J9 VM's portability library. I started writing a proof of concept implementation for this, and I'll add it to the wiki as soon as it's finished. I look forward to seeing the proof of concept. Were you thinking of introducing versioning and dependency management style functions? It would be interesting to have several such proof-of-concept implementations of component models which we can study and the decide which to use. We could even write import mechanisms for the different component models so they can import and use components from another model too (of course this would normally imply reduced performance). Regards, David. [1] http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200509.mbox/[EMAIL PROTECTED] -- David Tanzer, Haghofstr. 29, A-3352 St. Peter/Au, Austria/Europe http://deltalabs.at -- http://dev.guglhupf.net -- http://guglhupf.net My PGP Public Key: http://guglhupf.net/david/david.asc -- Brain: Pinky, I am in considerable pain. Pinky: Narf! Zort! Poit! Gat! I'm with you, Brain! -- Twas the Day Before Christmas signature.asc Description: This is a digitally signed message part
Re: [arch] VMCore / Component Model
Excellent - thanks On Sep 26, 2005, at 2:23 PM, David Tanzer wrote: I have now added the prototype implementation to JIRA: http://issues.apache.org/jira/browse/HARMONY-5 Cheers, David. On Mon, 2005-09-19 at 19:46 +0200, David Tanzer wrote: Today I have added some additional Information to the Wiki page. We should also consider using C++ and abstract classes to maintain our component model. While this would make inter-component communication slightly slower it would be easier to maintain. We should also think about using an existing component model like OSGi. The model I posted provides pretty fast communication between components without sacrificing too much flexibility, but it is maybe not as easy to maintain as a clean, object-oriented implementation (i.e. C++). We could discuss how important these aspects are to us, i.e. how much runtime efficiency we are willing to sacrifice for maintainability and flexibility and vice-versa. Regards, David. On Fri, 2005-09-16 at 21:47 +0200, David Tanzer wrote: Ok, it took a little bit longer than I first expected, but now my proof-of-concept implementation of the component model I described is available in the wiki: http://wiki.apache.org/harmony/ComponentModelFunctionPointers I have also linked it from the harmony architecture page. It contains a mechanism for loading components and a basic versioning and dependency management mechanism. The test case loads two components, where one depends on the other. I'll add some more explanations to the wiki page when I have more time, hopefully at the weekend. I have made several assumptions about the directory structure, the coding conventions and the documentation conventions, we should maybe discuss this in a different thread. Regards, David. On Tue, 2005-09-13 at 17:27 +0100, Tim Ellison wrote: David Tanzer wrote: Since we already started to define some component interfaces I think we also should start thinking about a component model which loads / connects such components. Maybe there are also some existing solutions we might want to look at (I must confess I didn't really search yet). Agreed, plus managing the API itself to ensure forwards/backwards version compatibility. I guess a requirement for such a component manager would be that it can load and connect components at runtime and that the specific implementations which are loaded can be configured. It might also be good if the same component implementations can be linked at compile time (i.e. statically) which could have benefits on embedded platforms, but I'm not sure if we really need this. I'm assuming that you are speculating on component management beyond the platform support (i.e. DLL-hell). The java world is already on this path with the OSGi framework (e.g. Felix). It will require a non- trivial solution to ensure that the runtime flexibility does not incur an unacceptable runtime cost. Another requirement would be that the components can be written in different programming languages (i.e. C, C++, Java, ...). It isn't really a problem to solve this for C and C++, but can we also easily support other programming languages? A simple way to implement such a component model in C would be an approach similar to the one Tim Ellison described in [1] where he explains the structure of the J9 VM's portability library. I started writing a proof of concept implementation for this, and I'll add it to the wiki as soon as it's finished. I look forward to seeing the proof of concept. Were you thinking of introducing versioning and dependency management style functions? It would be interesting to have several such proof-of-concept implementations of component models which we can study and the decide which to use. We could even write import mechanisms for the different component models so they can import and use components from another model too (of course this would normally imply reduced performance). Regards, David. [1] http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/ 200509.mbox/[EMAIL PROTECTED] -- David Tanzer, Haghofstr. 29, A-3352 St. Peter/Au, Austria/Europe http://deltalabs.at -- http://dev.guglhupf.net -- http://guglhupf.net My PGP Public Key: http://guglhupf.net/david/david.asc -- Brain: Pinky, I am in considerable pain. Pinky: Narf! Zort! Poit! Gat! I'm with you, Brain! -- Twas the Day Before Christmas -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] VMCore / Component Model
On Tue, 2005-09-20 at 08:40 -0400, Geir Magnusson Jr. wrote: On Sep 20, 2005, at 12:26 AM, Robin Garner wrote: I think it's important not to take Tim's point about performance too lightly here. There are some key interfaces between components that can't afford the overhead of a function call, let alone an indirect call via a function pointer. Three instances that come to mind are: - Allocation. For best performance, the common case of a new() needs to be inlined directly into the compiled code. - Write barrier (and read barrier if we need one). The common case of a write barrier should be a handful of instructions. - Yield points. Again should inline down to a couple of instructions. I'd be interested in any approaches you may have thought of for these interfaces. Are these things the components would worry about, or can an optimizer deal with these later if possible? I believe these are so performance critical that a design needs to be in place up front. If Harmony is to be competitive in performance terms with the current production VMs, we need to make sure that allocation, barriers, locking etc are as fast as possible. A design that adds an extra memory reference to one of these operations would not be competitive. Now, I'm really just making this up as I go along... what some people call thinking out loud, but I won't grace this with the term thinking. I've never written or been inside VM internals, so I'm inventing out of whole cloth here... In earlier discussions of componentization, I kept imagining a model where we have a defined set of capabilities that a component could optionally implement. There would be a required set, as well as optional ones. (This thinking is inspired by the old MSFT COM infrastructure...) In the case of a memory manager, a required one would be the interface containing the function pointers for a memory management. An optional one would be NativeInlineConversion or something, where an optimizer could find a new() and if it doesn't support the native inlineing, use the function call into the component, and if it does, ask the component for the bit of complied code to use. I think this is probably do-able, although for some things it would be reasonable to require the component to provide the inlineable code. The 'native code' passed back would have to be one level of compiler IR (internal representation). Unfortunately this has the side effect of making the compiler's IR public, breaking the modularity of the compiler(s) and to an extent distributing chunks of the compiler into other components. This is much less of a problem when doing Java in Java, but that's another thread. Actually, the more I think about this, the less I see the value in designing a component structure that allows run-time configurability. I believe compile-time configurability is the thing to aim for. Achieving good levels of abstraction without sacrificing performance is a difficult enough job at compile time in any case. Consider for example the design of the object model, and say that you want to support: 1) A real-time oriented incremental garbage collector 2) A mark-sweep collector for memory constrained environments 3) A generational reference counting collector as the general purpose high performance configuration. Collector 1) (if say you used a Brooks-style barrier) requires an additional word in the object header, to store a forwarding pointer for all objects, alive and dead. 2) requires only 1 bit in the object header (for a mark bit) and could conceivably steal a low-order bit from the class pointer (TIB in JikesRVM terms). 3) requires an extra word (or at least several extra bits) in mature objects, and gets better performance if nursery objects don't need that extra word. The object header also needs to store metadata associated with locking, address-based hashing and the TIB (pointer to per-class information). There is a complex tradeoff between size and encoding of information in the header word (see, for example, http://www.research.ibm.com/people/d/dgrove/papers/ecoop02.html), and since different layouts are possible depending on the different implementations selected. Much of the code in the runtime system needs to know how the object headers are laid out, and to access the metadata critical to them very rapidly. For example, a thin lock takes 5-10 instructions to acquire (in the frequent case). Adding (for example) a table lookup to find where in the object header the system had dynamically encoded the lock field would be disastrous for locking performance. So at least for modules that are critical to the performance of compiled code, I believe runtime configurability will be dogged with performance problems. Compilers, classloaders etc - fine, but the core runtime components - no. cheers Robin
Code into SVN, not the WIKI (Re: [arch] VMCore / Component Model)
Lets not store code in the wiki, but rather SVN. There's no control on a WIKI, so we have no clue what it is or really where it came from. As you know, we are being very careful about code pedigree for all sorts of good reasons. If you would like to get this code into SVN so others can start tweaking and playing, we should do that. 1) To get started, first look at the Authorized Contributor Questionnaire (ACQ) http://incubator.apache.org/harmony/auth_cont_quest.html or http://incubator.apache.org/harmony/auth_cont_quest.txt and fill out a copy, print it, and sign it. Fax to +1 203 665 6400 (that's my fax #). 2) Fill out an ICLA as required by part IV of the ACQ above http://www.apache.org/licenses/icla.txt print it, and sign it. Fax to same number. Assuming that all is well with the ACQ, this means that we can accept the code you have put in the WIKI into SVN for people to start playing with. You will have to add the code to a JIRA entry for the project, so you can definitively offer it under the Apache license. Note that we are going to be testing the contribution process that we have thought out, so please be patient :) geir On Sep 16, 2005, at 3:47 PM, David Tanzer wrote: Ok, it took a little bit longer than I first expected, but now my proof-of-concept implementation of the component model I described is available in the wiki: http://wiki.apache.org/harmony/ComponentModelFunctionPointers I have also linked it from the harmony architecture page. It contains a mechanism for loading components and a basic versioning and dependency management mechanism. The test case loads two components, where one depends on the other. I'll add some more explanations to the wiki page when I have more time, hopefully at the weekend. I have made several assumptions about the directory structure, the coding conventions and the documentation conventions, we should maybe discuss this in a different thread. Regards, David. On Tue, 2005-09-13 at 17:27 +0100, Tim Ellison wrote: David Tanzer wrote: Since we already started to define some component interfaces I think we also should start thinking about a component model which loads / connects such components. Maybe there are also some existing solutions we might want to look at (I must confess I didn't really search yet). Agreed, plus managing the API itself to ensure forwards/backwards version compatibility. I guess a requirement for such a component manager would be that it can load and connect components at runtime and that the specific implementations which are loaded can be configured. It might also be good if the same component implementations can be linked at compile time (i.e. statically) which could have benefits on embedded platforms, but I'm not sure if we really need this. I'm assuming that you are speculating on component management beyond the platform support (i.e. DLL-hell). The java world is already on this path with the OSGi framework (e.g. Felix). It will require a non-trivial solution to ensure that the runtime flexibility does not incur an unacceptable runtime cost. Another requirement would be that the components can be written in different programming languages (i.e. C, C++, Java, ...). It isn't really a problem to solve this for C and C++, but can we also easily support other programming languages? A simple way to implement such a component model in C would be an approach similar to the one Tim Ellison described in [1] where he explains the structure of the J9 VM's portability library. I started writing a proof of concept implementation for this, and I'll add it to the wiki as soon as it's finished. I look forward to seeing the proof of concept. Were you thinking of introducing versioning and dependency management style functions? It would be interesting to have several such proof-of-concept implementations of component models which we can study and the decide which to use. We could even write import mechanisms for the different component models so they can import and use components from another model too (of course this would normally imply reduced performance). Regards, David. [1] http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/ 200509.mbox/[EMAIL PROTECTED] -- David Tanzer, Haghofstr. 29, A-3352 St. Peter/Au, Austria/Europe http://deltalabs.at -- http://dev.guglhupf.net -- http://guglhupf.net My PGP Public Key: http://guglhupf.net/david/david.asc -- METHODEN DER BEWEISFHRUNG -- WISCHWEG - METHODE Man wischt die entscheidenden Stellen des Beweises sofort nach dem Anschreiben wieder weg (rechts schreiben, links wischen). -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: Code into SVN, not the WIKI (Re: [arch] VMCore / Component Model)
Geir Magnusson Jr. wrote: Lets not store code in the wiki, but rather SVN. There's no control on a WIKI, so we have no clue what it is or really where it came from. I totally agree with that, I just didn't know if I have SVN write access and how we structure the repository. I guess it would be good to set up a playground or sandbox where we can play around with prototypes like the one I've provided. As you know, we are being very careful about code pedigree for all sorts of good reasons. If you would like to get this code into SVN so others can start tweaking and playing, we should do that. 1) To get started, first look at the Authorized Contributor Questionnaire (ACQ) http://incubator.apache.org/harmony/auth_cont_quest.html or http://incubator.apache.org/harmony/auth_cont_quest.txt and fill out a copy, print it, and sign it. Fax to +1 203 665 6400 (that's my fax #). 2) Fill out an ICLA as required by part IV of the ACQ above http://www.apache.org/licenses/icla.txt print it, and sign it. Fax to same number. I have sent both documents via snail mail to Apache Software Foundation 1901 Munsey Drive, Forest Hill, MD 21050-2747 U.S.A. should I still fax it to you anyway? Assuming that all is well with the ACQ, this means that we can accept the code you have put in the WIKI into SVN for people to start playing with. You will have to add the code to a JIRA entry for the project, so you can definitively offer it under the Apache license. Note that we are going to be testing the contribution process that we have thought out, so please be patient :) Sure, no problem. I just thought we should start talking about code rather than abstract ideas. Regards, David. geir On Sep 16, 2005, at 3:47 PM, David Tanzer wrote: [Snip]
Re: Code into SVN, not the WIKI (Re: [arch] VMCore / Component Model)
On Sep 20, 2005, at 3:07 AM, David Tanzer wrote: Geir Magnusson Jr. wrote: Lets not store code in the wiki, but rather SVN. There's no control on a WIKI, so we have no clue what it is or really where it came from. I totally agree with that, I just didn't know if I have SVN write access and how we structure the repository. I guess it would be good to set up a playground or sandbox where we can play around with prototypes like the one I've provided. This will be a way to help work that out. As you know, we are being very careful about code pedigree for all sorts of good reasons. If you would like to get this code into SVN so others can start tweaking and playing, we should do that. 1) To get started, first look at the Authorized Contributor Questionnaire (ACQ) http://incubator.apache.org/harmony/auth_cont_quest.html or http://incubator.apache.org/harmony/auth_cont_quest.txt and fill out a copy, print it, and sign it. Fax to +1 203 665 6400 (that's my fax #). 2) Fill out an ICLA as required by part IV of the ACQ above http://www.apache.org/licenses/icla.txt print it, and sign it. Fax to same number. I have sent both documents via snail mail to Apache Software Foundation 1901 Munsey Drive, Forest Hill, MD 21050-2747 U.S.A. should I still fax it to you anyway? Yes. Assuming that all is well with the ACQ, this means that we can accept the code you have put in the WIKI into SVN for people to start playing with. You will have to add the code to a JIRA entry for the project, so you can definitively offer it under the Apache license. Note that we are going to be testing the contribution process that we have thought out, so please be patient :) Sure, no problem. I just thought we should start talking about code rather than abstract ideas. That's perfect. Regards, David. geir On Sep 16, 2005, at 3:47 PM, David Tanzer wrote: [Snip] -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] VMCore / Component Model
On Sep 13, 2005, at 1:47 PM, David Tanzer wrote: On Tue, 2005-09-13 at 17:27 +0100, Tim Ellison wrote: [Snip] I guess a requirement for such a component manager would be that it can load and connect components at runtime and that the specific implementations which are loaded can be configured. It might also be good if the same component implementations can be linked at compile time (i.e. statically) which could have benefits on embedded platforms, but I'm not sure if we really need this. I'm assuming that you are speculating on component management beyond the platform support (i.e. DLL-hell). The java world is already on this path with the OSGi framework (e.g. Felix). It will require a non-trivial solution to ensure that the runtime flexibility does not incur an unacceptable runtime cost. [Snip] I look forward to seeing the proof of concept. Were you thinking of introducing versioning and dependency management style functions? I was thinking about a really simple model, basically only a framework which loads shared libraries and creates the function pointer tables you described in your posting. Versioning and dependency management will be a requirement, but I didn't consider it so far in my proof-of-concept (which will take a few more days to finish), and the libraries which are loaded should be configurable. I also think we should be careful not to introduce too much runtime costs, and I'm implementing my prototype under the assumption that we don't need to manage arbitrary components (so I assume that the component interfaces and dependencies are known at compile time). Anyway it would be possible to extend my approach for arbitrary (i.e. unknown) components. I think that ultimately, this will be necessary. I imagine we want to be able to allow others to plug in new components easily. geir -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] VMCore / Component Model
Have you tried to communicate between two components, one in C(++) and one in Java? geir On Sep 19, 2005, at 1:46 PM, David Tanzer wrote: Today I have added some additional Information to the Wiki page. We should also consider using C++ and abstract classes to maintain our component model. While this would make inter-component communication slightly slower it would be easier to maintain. We should also think about using an existing component model like OSGi. The model I posted provides pretty fast communication between components without sacrificing too much flexibility, but it is maybe not as easy to maintain as a clean, object-oriented implementation (i.e. C++). We could discuss how important these aspects are to us, i.e. how much runtime efficiency we are willing to sacrifice for maintainability and flexibility and vice-versa. Regards, David. On Fri, 2005-09-16 at 21:47 +0200, David Tanzer wrote: Ok, it took a little bit longer than I first expected, but now my proof-of-concept implementation of the component model I described is available in the wiki: http://wiki.apache.org/harmony/ComponentModelFunctionPointers I have also linked it from the harmony architecture page. It contains a mechanism for loading components and a basic versioning and dependency management mechanism. The test case loads two components, where one depends on the other. I'll add some more explanations to the wiki page when I have more time, hopefully at the weekend. I have made several assumptions about the directory structure, the coding conventions and the documentation conventions, we should maybe discuss this in a different thread. Regards, David. On Tue, 2005-09-13 at 17:27 +0100, Tim Ellison wrote: David Tanzer wrote: Since we already started to define some component interfaces I think we also should start thinking about a component model which loads / connects such components. Maybe there are also some existing solutions we might want to look at (I must confess I didn't really search yet). Agreed, plus managing the API itself to ensure forwards/backwards version compatibility. I guess a requirement for such a component manager would be that it can load and connect components at runtime and that the specific implementations which are loaded can be configured. It might also be good if the same component implementations can be linked at compile time (i.e. statically) which could have benefits on embedded platforms, but I'm not sure if we really need this. I'm assuming that you are speculating on component management beyond the platform support (i.e. DLL-hell). The java world is already on this path with the OSGi framework (e.g. Felix). It will require a non-trivial solution to ensure that the runtime flexibility does not incur an unacceptable runtime cost. Another requirement would be that the components can be written in different programming languages (i.e. C, C++, Java, ...). It isn't really a problem to solve this for C and C++, but can we also easily support other programming languages? A simple way to implement such a component model in C would be an approach similar to the one Tim Ellison described in [1] where he explains the structure of the J9 VM's portability library. I started writing a proof of concept implementation for this, and I'll add it to the wiki as soon as it's finished. I look forward to seeing the proof of concept. Were you thinking of introducing versioning and dependency management style functions? It would be interesting to have several such proof-of-concept implementations of component models which we can study and the decide which to use. We could even write import mechanisms for the different component models so they can import and use components from another model too (of course this would normally imply reduced performance). Regards, David. [1] http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/ 200509.mbox/[EMAIL PROTECTED] -- David Tanzer, Haghofstr. 29, A-3352 St. Peter/Au, Austria/Europe http://deltalabs.at -- http://dev.guglhupf.net -- http://guglhupf.net My PGP Public Key: http://guglhupf.net/david/david.asc -- AUFGABEN DER PHYSIK -- QUANTENMECHANIK Gegebene Konstante: m(Kuh)=400 kg Die Kuh befinde sich auf einer Weide, die ringsum durch einen Zaun abgegrenzt ist. Der Weidezaun sie ideal gebaut, sodass die Kuh ihn (klassich gesehen) nicht passieren kann. Begrnden Sie, dass man die Kuh trotzdem mit gewisser Wahrscheinlichkeit ausserhalb der Weide antrifft. -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] VMCore / Component Model
On Sep 20, 2005, at 12:26 AM, Robin Garner wrote: I think it's important not to take Tim's point about performance too lightly here. There are some key interfaces between components that can't afford the overhead of a function call, let alone an indirect call via a function pointer. Three instances that come to mind are: - Allocation. For best performance, the common case of a new() needs to be inlined directly into the compiled code. - Write barrier (and read barrier if we need one). The common case of a write barrier should be a handful of instructions. - Yield points. Again should inline down to a couple of instructions. I'd be interested in any approaches you may have thought of for these interfaces. Are these things the components would worry about, or can an optimizer deal with these later if possible? Now, I'm really just making this up as I go along... what some people call thinking out loud, but I won't grace this with the term thinking. I've never written or been inside VM internals, so I'm inventing out of whole cloth here... In earlier discussions of componentization, I kept imagining a model where we have a defined set of capabilities that a component could optionally implement. There would be a required set, as well as optional ones. (This thinking is inspired by the old MSFT COM infrastructure...) In the case of a memory manager, a required one would be the interface containing the function pointers for a memory management. An optional one would be NativeInlineConversion or something, where an optimizer could find a new() and if it doesn't support the native inlineing, use the function call into the component, and if it does, ask the component for the bit of complied code to use. I'll sketch these out in code once we get David's code contributed properly into JIRA and we register his ACQ. On Mon, 2005-09-19 at 19:46 +0200, David Tanzer wrote: Today I have added some additional Information to the Wiki page. We should also consider using C++ and abstract classes to maintain our component model. While this would make inter-component communication slightly slower it would be easier to maintain. We should also think about using an existing component model like OSGi. The model I posted provides pretty fast communication between components without sacrificing too much flexibility, but it is maybe not as easy to maintain as a clean, object-oriented implementation (i.e. C++). We could discuss how important these aspects are to us, i.e. how much runtime efficiency we are willing to sacrifice for maintainability and flexibility and vice-versa. First order of priority, IMO, is to decide what are the performance critical components, and what are not. For the infrequent cases, we can afford a heavy-weight approach, for those that happen every few bytecodes, we need to be lean and mean. Is this the kind of optimization you want to do now, or do we want to put together a structure that supports it for later? geir -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] VMCore / Component Model
Peter Edworthy wrote: [Snip] First of all thanks to David Tanzer, having something solid to start from makes a tremendous difference. The code captures a basic functional spec for a component loader very well. I'm not very familiar with UNIX dynamic libs so if this is way off please say; It seems dlfcn is quite platform specific; would ltdl make more sense? Maybe. I just saw that the APR also provides methods for doing this, so that would be a good solution too. As i said, my code should only be a basis for discussion and a proof-of-concept. It seems as though only one 'native method' is provided per file this way, is that just for the demo code or is it a limitation of this method? No, it's not a limitation of the method, the struct TestComponent* could store an arbitrary number of functions. Am I right in thinking that to use the table of 'native method' pointers we will have to place the operands on the stack and then jump to the method (just checking as the code to do that would also be a component and so how would we bootstrap it, jikes had a clever method involving writing from a JVM a startup JVM image with the native code in it)? I'm not sure if we are talking about the same thing here, maybe I just fail to see some important part of the bigger picture. My intention was to provide a mechanism with which we could handle components like the ones for which Weldon Washburn wrote the interfaces for. For example, it could be used to load implementations of http://wiki.apache.org/harmony-data/attachments/HarmonyArchitecture/attachments/gc_interface.txt and http://wiki.apache.org/harmony-data/attachments/HarmonyArchitecture/attachments/vm_gc_interface.txt and interconnect them. It focuses on C (or C++) implementations so far, I'm not sure if we can easily extend it to support other programming languages. I think what you are talking about here is more like the light-weight native calls which where proposed by Mladen Turk (correct me if I'm wrong): http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200509.mbox/[EMAIL PROTECTED] Could this be implemented in Java so long as a native call mechanism existed to 'register' components with each other, there is probably no compelling reason to do this but it might improve cross platform support. That should be no big problem. I mentioned earlier that it would be cool if we had several different prototypes we can play with, compare to each other and discuss about, so maybe someone volunteers for writing one ;-) [Code-In lining] To have a JIT we must have a method of storing 'compiled code' and calling it we could create the basic components as native code stored in the JVM using same system as the JIT. The class loader could then mark inside the byte stream a call to this native code in place of the original byte code. Or the interpreter could act as though the byte code should be interpreted as a call to that method. If this is too inefficient then the JIT will compile the method that the call is within and at this point may well decide to in line the code from the method call. I don't see why we would want to or need to create a different in lining method for these aspects of the interpretor. If we are using the boot strapping method like jikes then maybe some methods should be JIT'ed before the image is written. [native code storage for the JIT] I've never tried to create a JIT, but I assume we need to consider some way of describing the side-effects of a section of code; i.e. what registers are changed/used as input and output. Or do JITs not normally need this as they compile whole methods and so use the stack for data in and out and assume all registers are dirty? Thanks in advance, Peter
Re: Code into SVN, not the WIKI (Re: [arch] VMCore / Component Model)
On Sep 20, 2005, at 9:11 AM, Davanum Srinivas wrote: Geir, When folks have sent in their ACQ/ICLA, we should give them direct commit access (after maybe a VOTE on the ppmc list). I really don't like putting so many road blocks, what exactly are we waiting for? What roadblocks are you talking about? We certainly want a vote and not just make everyone who fills in paperwork a committer. I don't think we need a high bar to entry, but at least a patch, maybe? This is a good subject to discuss. *Any* bulk contribution - i.e. code created outside of the day to day flow of the project by committers should come into a JIRA so the contribution can be inspected and understood to be a clearly delineated contribution. We will be keeping a record of all such contributions. geir Assuming that all is well with the ACQ, this means that we can accept the code you have put in the WIKI into SVN for people to start playing with. You will have to add the code to a JIRA entry for the project, so you can definitively offer it under the Apache license. Thanks, dims -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] VMCore / Component Model
Geir Magnusson Jr. wrote: Have you tried to communicate between two components, one in C(++) and one in Java? No, I didn't try that yet. For native compiled java code (gjc) I guess it should be straight forward to extend my proof-of-concept to support it. For java code which runs in the harmony vm the interface will have to be different, but we need this interface too. geir On Sep 19, 2005, at 1:46 PM, David Tanzer wrote: Today I have added some additional Information to the Wiki page. We should also consider using C++ and abstract classes to maintain our component model. While this would make inter-component communication slightly slower it would be easier to maintain. We should also think about using an existing component model like OSGi. The model I posted provides pretty fast communication between components without sacrificing too much flexibility, but it is maybe not as easy to maintain as a clean, object-oriented implementation (i.e. C++). We could discuss how important these aspects are to us, i.e. how much runtime efficiency we are willing to sacrifice for maintainability and flexibility and vice-versa. Regards, David. On Fri, 2005-09-16 at 21:47 +0200, David Tanzer wrote: Ok, it took a little bit longer than I first expected, but now my proof-of-concept implementation of the component model I described is available in the wiki: http://wiki.apache.org/harmony/ComponentModelFunctionPointers I have also linked it from the harmony architecture page. It contains a mechanism for loading components and a basic versioning and dependency management mechanism. The test case loads two components, where one depends on the other. I'll add some more explanations to the wiki page when I have more time, hopefully at the weekend. I have made several assumptions about the directory structure, the coding conventions and the documentation conventions, we should maybe discuss this in a different thread. Regards, David. On Tue, 2005-09-13 at 17:27 +0100, Tim Ellison wrote: David Tanzer wrote: Since we already started to define some component interfaces I think we also should start thinking about a component model which loads / connects such components. Maybe there are also some existing solutions we might want to look at (I must confess I didn't really search yet). Agreed, plus managing the API itself to ensure forwards/backwards version compatibility. I guess a requirement for such a component manager would be that it can load and connect components at runtime and that the specific implementations which are loaded can be configured. It might also be good if the same component implementations can be linked at compile time (i.e. statically) which could have benefits on embedded platforms, but I'm not sure if we really need this. I'm assuming that you are speculating on component management beyond the platform support (i.e. DLL-hell). The java world is already on this path with the OSGi framework (e.g. Felix). It will require a non-trivial solution to ensure that the runtime flexibility does not incur an unacceptable runtime cost. Another requirement would be that the components can be written in different programming languages (i.e. C, C++, Java, ...). It isn't really a problem to solve this for C and C++, but can we also easily support other programming languages? A simple way to implement such a component model in C would be an approach similar to the one Tim Ellison described in [1] where he explains the structure of the J9 VM's portability library. I started writing a proof of concept implementation for this, and I'll add it to the wiki as soon as it's finished. I look forward to seeing the proof of concept. Were you thinking of introducing versioning and dependency management style functions? It would be interesting to have several such proof-of-concept implementations of component models which we can study and the decide which to use. We could even write import mechanisms for the different component models so they can import and use components from another model too (of course this would normally imply reduced performance). Regards, David. [1] http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/ 200509.mbox/[EMAIL PROTECTED] -- David Tanzer, Haghofstr. 29, A-3352 St. Peter/Au, Austria/Europe http://deltalabs.at -- http://dev.guglhupf.net -- http://guglhupf.net My PGP Public Key: http://guglhupf.net/david/david.asc -- AUFGABEN DER PHYSIK -- QUANTENMECHANIK Gegebene Konstante: m(Kuh)=400 kg Die Kuh befinde sich auf einer Weide, die ringsum durch einen Zaun abgegrenzt ist. Der Weidezaun sie ideal gebaut, sodass die Kuh ihn (klassich gesehen) nicht passieren kann. Begrnden Sie, dass man die Kuh trotzdem mit gewisser Wahrscheinlichkeit ausserhalb der Weide antrifft.
Re: Code into SVN, not the WIKI (Re: [arch] VMCore / Component Model)
So let's do it then...Everyone interested should fill in their paperwork by end of the month. First week next month we can have a VOTE on the PPMC for each person based on their contributions so far. (Let each person state what they are bringing to the table as well if they haven't already). So by end of October we should have a roster of folks with commit privs who can then vote in the next set of committers (or as and when they want to). I really don't want to wait another 4 months and see that we are still in the same situation as we are in today. Thanks, dims On 9/20/05, Geir Magnusson Jr. [EMAIL PROTECTED] wrote: On Sep 20, 2005, at 9:11 AM, Davanum Srinivas wrote: Geir, When folks have sent in their ACQ/ICLA, we should give them direct commit access (after maybe a VOTE on the ppmc list). I really don't like putting so many road blocks, what exactly are we waiting for? What roadblocks are you talking about? We certainly want a vote and not just make everyone who fills in paperwork a committer. I don't think we need a high bar to entry, but at least a patch, maybe? This is a good subject to discuss. *Any* bulk contribution - i.e. code created outside of the day to day flow of the project by committers should come into a JIRA so the contribution can be inspected and understood to be a clearly delineated contribution. We will be keeping a record of all such contributions. geir Assuming that all is well with the ACQ, this means that we can accept the code you have put in the WIKI into SVN for people to start playing with. You will have to add the code to a JIRA entry for the project, so you can definitively offer it under the Apache license. Thanks, dims -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED] -- Davanum Srinivas : http://wso2.com/ - Oxygenating The Web Service Platform
Re: Code into SVN, not the WIKI (Re: [arch] VMCore / Component Model)
On Sep 20, 2005, at 9:36 AM, Davanum Srinivas wrote: So let's do it then...Everyone interested should fill in their paperwork by end of the month. First week next month we can have a VOTE on the PPMC for each person based on their contributions so far. (Let each person state what they are bringing to the table as well if they haven't already). So by end of October we should have a roster of folks with commit privs who can then vote in the next set of committers (or as and when they want to). I really don't want to wait another 4 months and see that we are still in the same situation as we are in today. We won't be. I'll post another note outlining what I think we should do, and we can agree. geir Thanks, dims On 9/20/05, Geir Magnusson Jr. [EMAIL PROTECTED] wrote: On Sep 20, 2005, at 9:11 AM, Davanum Srinivas wrote: Geir, When folks have sent in their ACQ/ICLA, we should give them direct commit access (after maybe a VOTE on the ppmc list). I really don't like putting so many road blocks, what exactly are we waiting for? What roadblocks are you talking about? We certainly want a vote and not just make everyone who fills in paperwork a committer. I don't think we need a high bar to entry, but at least a patch, maybe? This is a good subject to discuss. *Any* bulk contribution - i.e. code created outside of the day to day flow of the project by committers should come into a JIRA so the contribution can be inspected and understood to be a clearly delineated contribution. We will be keeping a record of all such contributions. geir Assuming that all is well with the ACQ, this means that we can accept the code you have put in the WIKI into SVN for people to start playing with. You will have to add the code to a JIRA entry for the project, so you can definitively offer it under the Apache license. Thanks, dims -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED] -- Davanum Srinivas : http://wso2.com/ - Oxygenating The Web Service Platform -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: Code into SVN, not the WIKI (Re: [arch] VMCore / Component Model)
On Sep 20, 2005, at 9:29 AM, [EMAIL PROTECTED] wrote: Dude, It's catch 22. There weren't any legitimate committers (because there was no initial code base) at the beginning of the project to vote. Because of this, there needs to be a lower barrier of entry than a formal procedure. Having a barrier to entry isn't in conflict with a formal procedure. We are going to have a formal procedure because the code pedigree is critical for this project. Otherwise I might suggest a segment of the willing go off and create an initial codebase in a CVS/SVN somewhere that is more open and then submit it. For a project with no code this seems a bit officious. Let the likely people in (Mladen for instance is an apache committer in good standing who has plans to do something) and then let Darwin sort them out. Then they can vote in committers in the normal way. The project needs code!!! Rather than being officious, the goal should be to facillitate every means possible to make that happen. It will be messy and there will be serveral misdirections but thats what a repository is for. Forward momentum. This is clearly a position we all believe in - we need code. I'll remark in another summary message. geir -Andy Geir Magnusson Jr. wrote: On Sep 20, 2005, at 9:11 AM, Davanum Srinivas wrote: Geir, When folks have sent in their ACQ/ICLA, we should give them direct commit access (after maybe a VOTE on the ppmc list). I really don't like putting so many road blocks, what exactly are we waiting for? What roadblocks are you talking about? We certainly want a vote and not just make everyone who fills in paperwork a committer. I don't think we need a high bar to entry, but at least a patch, maybe? This is a good subject to discuss. *Any* bulk contribution - i.e. code created outside of the day to day flow of the project by committers should come into a JIRA so the contribution can be inspected and understood to be a clearly delineated contribution. We will be keeping a record of all such contributions. geir Assuming that all is well with the ACQ, this means that we can accept the code you have put in the WIKI into SVN for people to start playing with. You will have to add the code to a JIRA entry for the project, so you can definitively offer it under the Apache license. Thanks, dims -- Andrew C. Oliver SuperLink Software, Inc. Java to Excel using POI http://www.superlinksoftware.com/services/poi Commercial support including features added/implemented, bugs fixed. -- Geir Magnusson Jr +1-203-665-6437 [EMAIL PROTECTED]
Re: [arch] VMCore / Component Model
I think it's important not to take Tim's point about performance too lightly here. There are some key interfaces between components that can't afford the overhead of a function call, let alone an indirect call via a function pointer. Three instances that come to mind are: - Allocation. For best performance, the common case of a new() needs to be inlined directly into the compiled code. - Write barrier (and read barrier if we need one). The common case of a write barrier should be a handful of instructions. - Yield points. Again should inline down to a couple of instructions. I'd be interested in any approaches you may have thought of for these interfaces. On Mon, 2005-09-19 at 19:46 +0200, David Tanzer wrote: Today I have added some additional Information to the Wiki page. We should also consider using C++ and abstract classes to maintain our component model. While this would make inter-component communication slightly slower it would be easier to maintain. We should also think about using an existing component model like OSGi. The model I posted provides pretty fast communication between components without sacrificing too much flexibility, but it is maybe not as easy to maintain as a clean, object-oriented implementation (i.e. C++). We could discuss how important these aspects are to us, i.e. how much runtime efficiency we are willing to sacrifice for maintainability and flexibility and vice-versa. First order of priority, IMO, is to decide what are the performance critical components, and what are not. For the infrequent cases, we can afford a heavy-weight approach, for those that happen every few bytecodes, we need to be lean and mean. cheers Robin
Re: [arch] VMCore / Component Model
Ok, it took a little bit longer than I first expected, but now my proof-of-concept implementation of the component model I described is available in the wiki: http://wiki.apache.org/harmony/ComponentModelFunctionPointers I have also linked it from the harmony architecture page. It contains a mechanism for loading components and a basic versioning and dependency management mechanism. The test case loads two components, where one depends on the other. I'll add some more explanations to the wiki page when I have more time, hopefully at the weekend. I have made several assumptions about the directory structure, the coding conventions and the documentation conventions, we should maybe discuss this in a different thread. Regards, David. On Tue, 2005-09-13 at 17:27 +0100, Tim Ellison wrote: David Tanzer wrote: Since we already started to define some component interfaces I think we also should start thinking about a component model which loads / connects such components. Maybe there are also some existing solutions we might want to look at (I must confess I didn't really search yet). Agreed, plus managing the API itself to ensure forwards/backwards version compatibility. I guess a requirement for such a component manager would be that it can load and connect components at runtime and that the specific implementations which are loaded can be configured. It might also be good if the same component implementations can be linked at compile time (i.e. statically) which could have benefits on embedded platforms, but I'm not sure if we really need this. I'm assuming that you are speculating on component management beyond the platform support (i.e. DLL-hell). The java world is already on this path with the OSGi framework (e.g. Felix). It will require a non-trivial solution to ensure that the runtime flexibility does not incur an unacceptable runtime cost. Another requirement would be that the components can be written in different programming languages (i.e. C, C++, Java, ...). It isn't really a problem to solve this for C and C++, but can we also easily support other programming languages? A simple way to implement such a component model in C would be an approach similar to the one Tim Ellison described in [1] where he explains the structure of the J9 VM's portability library. I started writing a proof of concept implementation for this, and I'll add it to the wiki as soon as it's finished. I look forward to seeing the proof of concept. Were you thinking of introducing versioning and dependency management style functions? It would be interesting to have several such proof-of-concept implementations of component models which we can study and the decide which to use. We could even write import mechanisms for the different component models so they can import and use components from another model too (of course this would normally imply reduced performance). Regards, David. [1] http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200509.mbox/[EMAIL PROTECTED] -- David Tanzer, Haghofstr. 29, A-3352 St. Peter/Au, Austria/Europe http://deltalabs.at -- http://dev.guglhupf.net -- http://guglhupf.net My PGP Public Key: http://guglhupf.net/david/david.asc -- METHODEN DER BEWEISFHRUNG -- WISCHWEG - METHODE Man wischt die entscheidenden Stellen des Beweises sofort nach dem Anschreiben wieder weg (rechts schreiben, links wischen). signature.asc Description: This is a digitally signed message part
Re: [arch] VMCore / Component Model
David Tanzer wrote: Since we already started to define some component interfaces I think we also should start thinking about a component model which loads / connects such components. Maybe there are also some existing solutions we might want to look at (I must confess I didn't really search yet). Agreed, plus managing the API itself to ensure forwards/backwards version compatibility. I guess a requirement for such a component manager would be that it can load and connect components at runtime and that the specific implementations which are loaded can be configured. It might also be good if the same component implementations can be linked at compile time (i.e. statically) which could have benefits on embedded platforms, but I'm not sure if we really need this. I'm assuming that you are speculating on component management beyond the platform support (i.e. DLL-hell). The java world is already on this path with the OSGi framework (e.g. Felix). It will require a non-trivial solution to ensure that the runtime flexibility does not incur an unacceptable runtime cost. Another requirement would be that the components can be written in different programming languages (i.e. C, C++, Java, ...). It isn't really a problem to solve this for C and C++, but can we also easily support other programming languages? A simple way to implement such a component model in C would be an approach similar to the one Tim Ellison described in [1] where he explains the structure of the J9 VM's portability library. I started writing a proof of concept implementation for this, and I'll add it to the wiki as soon as it's finished. I look forward to seeing the proof of concept. Were you thinking of introducing versioning and dependency management style functions? It would be interesting to have several such proof-of-concept implementations of component models which we can study and the decide which to use. We could even write import mechanisms for the different component models so they can import and use components from another model too (of course this would normally imply reduced performance). Regards, David. [1] http://mail-archives.apache.org/mod_mbox/incubator-harmony-dev/200509.mbox/[EMAIL PROTECTED] -- Tim Ellison ([EMAIL PROTECTED]) IBM Java technology centre, UK.
