In the JGDMS fork of River, in addition to Reggie's proxy returing unmarshalled service proxy's from the ServiceRegistrar lookup method, SafeServiceRegistrar provides a lookup method that returns instances of java.lang.reflect.Proxy that implement interfaces to retrieve the service, it's attributes and it's codebase uri and signer certs.
The client can authenticate, filter and grant permissions (for deserialization and codebase download) before retrieving the service proxy using a method call. In this case the service proxy is obtained by from the reflection proxy instead of MarshalledIstance. Cheers, Peter. Sent from my Samsung device. Include original message ---- Original message ---- From: "Michał Kłeczek (XPro Sp. z o. o.)" <michal.klec...@xpro.biz> Sent: 08/02/2017 05:51:07 am To: dev@river.apache.org Subject: Re: OSGi - deserialization remote invocation strategy So I must have misunderstood the part about smart proxies being obtained via "reflection proxies" or MarshalledInstances. What are these "reflection proxies"? Thanks, Michal Peter wrote: No, no bootstrap objects. Cheers, Peter. Sent from my Samsung device. Include original message ---- Original message ---- From: "Michał Kłeczek (XPro Sp. z o. o.)" <michal.klec...@xpro.biz> Sent: 08/02/2017 12:28:50 am To: dev@river.apache.org Subject: Re: OSGi - deserialization remote invocation strategy Are you proposing to provide a bootstrap object that will download some meta information prior to class resolution? How does it differ from simply changing annotations to be those "bootstrap objects" instead of Strings? Thanks, Michal Peter wrote: Proposed JERI OSGi class loading strategy during deserialization. Record caller context - this is the default bundle at the beginning of the stack. It is obtained by the InvocationHandler on the client side. The InvocationDispatcher on the server side has the calling context of the Remote implementation. The reflection dynamic proxy must be installed in the client's class loader, so the InvocationHandler knows exactly what it is, it will be passed to the MarshalInputStream. Any interfaces not found in the client's bundle can be safely shed. For a smart proxy the reflection proxy will be installed in the smart proxy loader. The smart proxy is obtained either via a reflection proxy or a MarshalledInstance. MarshalledInstance also passes in the callers loader to the MarshalInputStream. The smart proxy classloader is not a child loader of the clients loader, instead it's a bundle that imports service api packages, with a version range that overlaps those already imported by the client. Both Invocationhandler and InvocationDispatcher utilise MarshalInputStream and MarshalOutputStream, for marshalling parameters and return values. The codebase annotation bundle's manifest contains a list of package imports. Do we need to make a list of package imports for every new bundle that we load? Do we need to record the wiring and packages and their imports from the remote end? I don't think so, the bundles themselves contain this information, I think we just need to keep the view of available classes relevant to the current object being deserialized. Codebase Annotations are exact versions! They need to be to allow the service to ensure the correct proxy codebase is used. Other proxy codebases will be installed in the client, possibly different versions, but these won't be visible through the resolved dependencies, because the proxy codebases only import packages at the client and OSGi restricts visibility to the current bundle's own classes and any imported packages. Instead of appending dependencies to the codebase annotation they'll need be defined in the proxy's bundle manifest. Of course if an identical version of a proxy codebase bundle is already installed at the client, this will be used again. Because a bundle generally imports packages (importing entire bundles is discouraged in OSGi), there may be classes that aren't visible from those bundles, such as transient imports, but also including private packages that aren't exported, private implementations need to be deserialized, but is it possible to do so safely, without causing package conflicts? Private implementation classes can be used as fields within an exported public object, but cannot and should not escape their private scope, doing so risks them being resolved to a bundle with the version of the remote end, instead of the locally resolved / wired package, causing ClassClassExceptions. Initial (naive) first pass strategy of class resolution (for each branch in the serialized object graph)?: 1. Try current bundle on the stack (which will be the callers bundle if we haven't loaded any new bundles yet). 2. Then use the package name of a class to determine if the package is loaded by any of the bundles referenced by the callers bundle imports (to handle any private implementation packages that aren't in the current imports). Is this a good idea? Or should we go straight to step 3 and let the framework resolve common classes, what if we use a different version to the client's imported bundle? Should we first compare our bundle annotation to the currently imported bundles and select one of those if it's a compatible version? Yes, this could be an application bundle, otherwise goto 3. 3. Load bundle from annotation (if already loaded, it will be an exact version match). Place the new bundle on top of the bundle stack, remove this bundle from the stack once all fields of this object have been deserialized, returning to the previous bundle context. We are relying on the current bundle to wire itself up to the same package versions of the clients bundle imports, for shared classes. Classes that use different bundles will not be visible to the client, but will need to be visible to the current object's bundle. 4. Place a bundle reference on the stack when a new object is deserialized from the stream and remove it once all fields have been deserialized. (we might need to remember stack depth). 5. Don't place non bundle references on the stack. For example system class loader or any other class loader, we want resolution to occur via the OSGi resolution process. What about a simpler strategy (again naive), where we don't attempt to resolve private implementation classes? 1. The calling class' bundle, is given priority. 2. Load bundle from annotation (exact version), when not found in calling class. 3. No stack, what if an application bundle from server is loaded that conflicts with an existing bundle resolved by the client? 4. What about walking back through the stack? Probably unnecessary, as the containing object will reference the class by a common interface, the outer object may not need to reference it at all. But what if the outer object passed it in during construction? Revised strategy: 1. Attempt to load from current bundle on stack (the stack begins with the client's Bundle, each node in the graph has its bundle added to the stack and is also removed after that node is completely deserialized. 2. If unsuccessful, walk back through deserialized bundle reference stack and attempt to load class. Why not start at the beginning of the stack? We are expecting bundles to wire up to currently loaded versions, but bundles can import different package versions for implementation, safest to start with current bundle and consult parent if not found in the current bundle dependency graph, ie possibly passed in during object construction or an handback implemented in the client, from an earlier invocation or dependency injected. 3. The client is responsible for determining compatibility with the service api it's interested in from the Import Package Entry's, prior to unmarshalling a service proxy. 4. If a bundle previously on the stack resolves a class, then this object's bundle reference is placed on the top of the stack, it is removed once the current object and all it's fields have been completely deserialized. 5. Load bundle from annotation (exact version). 6. No attempt will be made to directly load from wired bundles, always rely on wires, otherwise we may utilise an incompatible package / bundle. Do we need a graph of the wiring from the remote end? During serialization (from the remote end) do we need to determine if a bundle has dependants and send some sort of version range information? When a class descriptor is read in from a stream, the class descriptor contains information about fields and it's serializable supertype class (if it exists) are also read in from the stream, before any field objects are read in, the declared field types are visible from the bundle of the current object being deserialized. The objects that will be assigned to those field types must also resolve to those types. Hence bundles being resolved as part of deserialization must favour already resolved packages for imports. What if a bundle requires a specific package version? This is why the bundle must be given first attempt to resolve an objects class and rely on the bundle dependency resolution process. OSGi must be allowed to wire up dependencies, we must avoid attempting to make decisions about compatibility and use the current bundle wires instead (our stack). The BundleReference stack is designed to follow the wires (dependency links between bundles), to allow private classes to be resolved, as they're not visible from other bundles. We can't rely on annotations to resolve private classes, because we can't predict the way bundle dependency's are resolved in remote JVM's. General recommendations for OSGi: * The service should use as wide a version range as possible for service api. * It is better to create new service api in a new bundle than to evolve in a backward compatible manner, as an incremental change may not be compatible if additional classes and methods are missing from the client, that the service proxy depends on. * Don't split packages. * Private implementation classes are ok, provided they remain within public exported classes and don't escape, otherwise they may not link up properly upon deserialization. * The proxy should minimise the package imports it uses. * There must be only one compatible service api version installed already in the client. * Duplicates of incompatible versions of service api are ok. The catch is, it may not be possible to build the bundle stack without some programming hooks in ObjectInputStream. Unfortunately we don't have any control over OIS, the necessary hooks could however be added to AtomicMarshalInputStream. Cheers, Peter.
