Maybe there are some misunderstanding somewhere... see below; On Wed, Feb 8, 2017 at 3:35 AM, Peter <j...@zeus.net.au> wrote: > I'm currently only considering OSGi server -> OSGi client. Mick's investigating all four options.
Ok, makes it a lot easier for me to follow. > Not expecting the client calling bundle to resolve everything, hence the > stack, so we have the full visibility of the bundle of the class that was > last resolved, so we can resolve its fields from it's bundle. Eg it > might import packages the client does not. What is "client calling bundle"? I assume that it is the bundle that contains the service lookup... First of all, is the OSGi integration intending to be at "OSGi service level" ? If so, the Jini support bundle(s) in OSGi, will monitor Jini service registrations and register those in the local OSGi registry and all clients would end up being rather ignorant about the remote lookup. It is also possible to "be involved" in the service lookup (and registration) through the Service Hook API (I might got the name of it wrong), so that only when services are requested or service listeners are registered, does the Jini support kick in. And this can therefore work both ways... So, that said; I agree that the "client calling bundle" does not resolve anything. And I would go further and say; IF the client calling bundle looks up Jini services directly, all bets are off. This introduces some workable constraints of what could happen. > The "exact version" thing (only applies to the proxy bundle as we > expect the framework to load its deps) can be relaxed to compatible > versions to increase class sharing if you think it helps. The proxy bundle > doesn't export anything at the client, only the server, it just seems to > make sense to keep the latest proxy communicating in case that last > bug fix release addresses a security issue. All proxy classes are > implementation only classes. Yes, correct. The 'temporary' proxy bundle should not export any packages. > Because the proxy bundle manifest declares version import ranges, > I'm expecting the framework to favour already loaded bundles to > satisfy package import deps. Uhhhh... The OSGi framework doesn't "auto load" bundles. It is an explicit step. There are many "bundle loaders" around, such as Apache Karaf "Features" and "deploy/" directory. Most frameworks can also be instructed to load bundles at boot. So there is no "favor already loaded bundles". If there is no bundle satisfying the Import-Package, the bundle being resolved will not go to RESOLVED state. If there are many bundles (quite typical) satisfying an Import-Package, with all its additional contraints (versions, attributes, uses, ...), then you enter the NP-complete problem that Michal mentioned, finding a combination of wiring that satisfy as many bundles as possible. This is a problem mainly due to "uses" (since we seldom use attributes), where graphs of types must end up matching in the class space. See (especially) section 3.7 in OSGi 6.0 Core Specification. If anyone can get their head around those details in the first pass, it is you Peter. Not easy reading... > If the client is matching service api with the correct import package > version ranges (requirements defined by entry's), the proxy bundle > should find the service api and other imported packages are already > loaded. Eg the client may use the requirements to use the resource > service or whatever the new bundle repository standard service is > called now to preload the requirements. The client may also perform > upgrades before downloading a service. I think this is a misunderstanding as well. By doing what I wrote in the beginning (listen on Jini Service Registrations and register "something" (Remote proxy or a local proxy) in the OSGi registry) then the client bundle doesn't need to know anything. Also, the Jini support bundle gets plenty of information both from the OSGi Registry as well as the Jini Registry. So, when something disappears from Reggie, remove it from the local Registry and vice versa. Now, the deserialization of the Reggie proxy should detect version changes and update a cache. And I think that Paremus idea of "Bundle Garbage Collection" is sound, but something for later discussion. Point being; No need to figure out what can and can not be unloaded. ALSO, since OSGi mandates intermittent service availability, most OSGi applications are reasonably capable to handle that the Jini service will "disappear" and is required to release any held references to the object(s) so that regular GC can toss out the classes and classloader when bundle is unloaded. > In the majority of cases I don't think there's going to be much state > in the smart proxy that can't be loaded via the smart proxy bundle > and it's package imports, except for the odd handback, which the > client bundle should have the opportunity to resolve before > resorting to using an annotation. Sorry, I don't understand this statement. > I'm not quite ready to agree it's too complex and it's unsolveable, I > think we should at least explore it and understand it before we junk > the idea of supporting OSGi. If we are talking osgi<->osgi, I think there is reasonable chance to succeed. > Rather than utilise the Java2 class loading I was planning to cast > ClassLoaders to BundleReference where appropriate and utilise > the Bundle. My gut says that this is not needed, if you go with my initial proposal. A Jini support bundle ends up having access to the BundleContext, from where everything else can be reached. > I did notice you're interpretation of what I've written is different than > mine, so I think I need to put some effort into communicating more > effectively. I think you're interpretation of codebase annotation > "version is fixed" ignores that the annotation is only consulted after > determining that the current class is not available in our Bundles > currently participating in deserialization. It doesn't apply to resolved > imported packages as annotations aren't used for them at all. OR, I have been burned in class resolution in OSGi enough times to have a feeling that it is more difficult than it seems. Any simple example I can think of would work... > For example, the first class we attempt to resolve during unmarshalling > belongs to a smart proxy, the client Bundle can't find the class. Ask the > framework to load the proxy bundle from the codebase annotation, it > does so and resolves all necessary package imports declared in its > manifest. We now continue deserializing the smart proxy class fields > with the visibility of the smart proxy's bundle. The smart proxy may > contain fields referencing objects resolved from its imports, we ensure > those classes deserialize their fields with the visibility of their own bundles. So, the smart proxy's "bundle" is a bundle on the server side as well? The smart proxy may also contain objects that are of a class that is not visible... And it may be N levels deep from the "field" in the smart proxy. > Every time we can't resolve a class we first check if it's a handback > or parameter from a preceeding object in the graph, thus we walk > our graphs bundle stack. This is probably the bit I don't understand at all. On one hand you want to depend on OSGi framework to do the resolution, but OTOH you have something called a "bundle stack"? What is that? > If we still haven't resolved a class only then do we load a bundle from > it's codebase annotation url and check it can be cast to the field > before assigning it. If it can't be cast to that field, we throw an exception. "assigning it"? Doesn't the code in reality looks something like; private Map map; private void readObject( ObjectInputStream in ) { map = (Map) in.readObject(); } where the read object may be of a class not visible to neither the smart proxy's classloader nor any helper? Maybe you meant that it will implicitly thrown just by the above code. > In the case of a non smart proxy, there is no codebase, deserialization > will be loaded by and rely completely on the visibility of the client bundle. > > I think OSGi will be a lot less dependant on annotations than say a std env. Possibly... I think that largely depend on the usecase. > Still I guess wiring may be an an option, so as Michael suggests, annotate objects with their wiring graphs. Ok, here is what I see being the issue at hand; you think that it is possible to delay the bundle resolution of packages until deserialization itself. I think that is not possible. There is a need to bring all "non-available classes" into the client (and I think as a bundle is the correct solution). So, pick up the bundle reference of the smart proxy, and the bundle wiring graph of it. When the smart proxy arrives on the client, before doing anything else, load the bundles on the client. After that, it should be a "local JVM" problem, both to deserialize the smart proxy as well as every object communicated over the network. > What would we be considering if we hadn't been pre exposed to codebase annotations? > Standard deserialization uses one classpath, each bundle has its own unique classpath. It is not only about classpath. It is about class space and visibility as well. And serialization needs to bypass visibility, just like it bypassed other constraints before (such as bypassing constructors and initializing final fields), and that is a separate issue than the bundle loading mechanism. Cheers -- Niclas Hedhman, Software Developer http://polygene.apache.org - New Energy for Java