Hi Michal,
Your code's included in svn at:
https://svn.apache.org/repos/asf/incubator/river/jtsk/skunk/river-modules
If you upload patches to Jira, I'll add them for you.
Cheers,
Peter.
Michal Kleczek wrote:
Thanks for taking time to look at this.
1. I am open to _any_ refactoring (and changing the code completely) - my code
is just a prototype. BTW - would it be possible to put it somewhere in SVN? It
would be much easier for me to contribute - I don't need a commit access -
just a simple way to produce patches. (The version attached to JIRA needs
patching badly since it does not work :) )
2. I am not sure I understand the compatibility issues. I assume we're talking
about a scenario where there is a running service that has MarshalledInstance
objects saved somewhere and we want to upgrade it?
I think it can be easily solved by making ModuleAnnotatedInputStream capable
of reading both String and Module annotations. String annotations could be
translated to DefaultRmiModule instances installed and used as usual later on.
I would even say that my version of RMIClassLoaderSpi should be changed so
that it can handle both serialized and encoded Modules as well as old style
String annotations (lists of URLs). It should also produce old style
annotations for DefaultRmiModules - otherwise tools such as VisualVM are not
going to work ( tested this :) )
3. I found the following code in River dependent on annotations being Strings
(and RMIClassLoader as well):
1) Anything that makes use of MarshalledObject (for example DiscoveryV1,
RemoteEventListener)
2) Reggie entries assume annotation is a string
3) Outrigger - I don't know the details - just checked it calls
RMIClassLoader.getClassAnnotation()
4) ProxyTrustVerifier - does a check if a candidate proxy's ClassLoader is a
proper RMI ClassLoader by comparing annotations produced by RMIClassLoader
Looks to me it is not straightforward to remove River dependency on
RMIClassLoader (and - to be honest - it was not my goal at all - I wanted to
have a piece of code that would plug into existing River)
4. We can easily support scenarios where a client uses old services (the ones
not aware of Modules) - see p. 2. But I don't thing there is an easy way for a
service that annotates it's objects with Modules to provide a way for an old
client to use it.
5. Didn't thing too much about implementing all this as a URL handler - it
looks pretty complicated to me - handling Manifest classpaths, multi URL
codebases etc.
6. Placing constraints on Modules is certainly possible but I have no idea how
useful it could be and how hard it would be to implement it.
The only constraint on a Module we need right now (so that the functionality
is on-pair with existing River) is Integrity.YES. We handle it by just having
a boolean argument in Module methods. It made implementation easy (since it is
similar to what we have in River right now). Also - I've got no idea how to
implement other constraints - we would have do something really smart in
PreferredClassLoader :)
7. I am not following on how you could get rid of CachingProxyTrust since you
need to verify two objects.
Thanks,
Michal
On Tuesday 02 of November 2010 14:52:57 Peter Firmstone wrote:
Thanks Michal, that was helpful, sorry for taking so long to reply.
Having chewed the details, I think now it is safe for me to comment, I'm
wondering if your open to some refactoring?
You've managed to produce a lot of code in a very short time, your
productivity is quite impressive and you seem proficient using Secure
Jini Services.
I've been thinking about your use of objects as annotations, which I
readily took to, something that's bothering me is backward
compatibility, an Object could be changed to a String, but a String is
final and existing implementations are stuck with it. MarshalledInstance
is part of the Discovery 2 Protocol, and Reggie, so it's serialized form
matters to remote foreign parties.
The current way that MarshalledInstance Serialized Form is implemented
leaves it open to expansion due to readObject's implementation. This
means we can add fields without breaking backward compatibility. The
earlier implementations can utilise the existing fields and drop any
superfluous objects they don't know about.
I'm wondering if it's possible to implement a Module URL scheme, similar
to Codebase Services?
I've also wondered if using an authenticating verifier proxy during
deserialization of MarshalledInstance would be enough, saving the need
to Cache using the caching proxy trust verifier. MarshalledInstance is
the point of contact for Discovery and Lookup, if we require
authentication and verification, then that might be sufficient for
MarshalledInstance.
We can utilise your Module annotated streams via Jeri, unbeknown to
external implementations.
I've attached a very rough example draft rudimentary untested
MarshalledInstance (TODO: server min principal authentication) which
uses defensive copying and message digests to confirm the deserialized
state as part of the private implementation of MarshalledInstance. This
makes object integrity the responsibility of the Object itself, rather
than an external mechanism. So in this case privacy is the
responsibility of external mechanisms, but object integrity is an
internal concern.
Gregg Wonderly created a CodebaseAnnotationClassloader (I think that's
the name I mentioned earlier), this might remove the need to use
RMIClassLoaderSPI and provide the opportunity to add some additional
functionality.
See below for some more comments:
Best Regards,
Peter.
Michal Kleczek wrote:
On Thursday 28 of October 2010 10:36:38 Peter Firmstone wrote:
Ok Interesting, anyone implementing a Service should be quite capable of
implementing equals, we should then specify that equals and hashcode
methods should be implemented for ProxyTrust.
For my benefit can you go over the process of how your new code works?
I'm interested in the choices you've made & why.
Sure.
Several choices were made because:
1. I wanted to reuse as much as possible from existing River
2. I wanted to have a working prototype fast :)
Anyway:
1. The main idea is to have annotations as objects that could be verified
using standard River proxy verification facilities. To be honest the
idea of having Module interface that enables plugging different
classloading mechanisms is something completely optional.
2. For the client the basic scenario is:
a) The client gets a serialized object
b) When it is deserialized annotations (Modules) are read and "installed"
c) Installing a Module means:
c1) checking if it was already installed
c2) if not - prepare it using VerifyingProxyPreparer (I've choosen not to
place any InvocationConstraints on Modules - I don't think it is
necessary)
I'm curious why client MethodConstraints aren't needed?
d) after a Module is installed it is used to load classes
3. There are several places in River that depend on annotations being
Strings provided by RMIClassLoader. The most important places are
a) Discovery
b) Reggie implementation
c) ProxyTrustVerifier
Since I did not want to modify this code I had to implement
RMIClassLoaderSpi so that it would provide serialized Modules as
Strings. I've choosen to simply serialize them to byte arrays and Base64
encode them.
4. There are two important Module implementations available:
a) DefaultRmiModule
b) ProxyTrustModule
DefaultRmiModule can be trusted by the client without contacting any
service - it uses RequireDlPermissionClassProvider to load classes.
ProxyTrustModule on the other hand uses a simple PreferredClassProvider
to load classes. It is a smart proxy that implements
getProxyTrustIterator() so that it can be verified by
ProxyTrustVerifier.
On the server before we can annotate our objects with modules we have to
register a Module that will be used as annotation.
If the server does not register any Module - DefaultRmiModule is going to
be used as annotation.
The server can register a ProxyTrustModule as annotation - to do that it
first must export an object that can be contacted to obtain a Module
verifier. In other words - a service must either:
a) export two ProxyTrusts (one for Modules and another one for the
service itself)
b) its ProxyTrust must provide a verifier that is able to verify both a
Module and a service proxy
To make it possible to use my code with existing services without
modifying them I've decided to implement a special ModuleExporter which
will override the service ServerProxyTrust implementation so that
getProxyVerifier will return a Verifier capable of verifying both a
Module and the service proxy. This verifier works as follows:
1. Check if an object being verified is a Module.
2. If so - delegate to a module verifier
3. If not it means we're verifying a service proxy so delegate to a
default verifier.
The problem is though that when this verifier is deserialized the module
that is capable of loading the service proxy verifier is not yet
installed (the client got this verifier to verify a Module before it can
load any classes). That's why I called it LazyCompositeVerifier - the
default verifier is not deserialized until it is actually needed.
The only problem left is that every time a client gets a service proxy it
will issue a remote call twice to get a verifier: first to get a
verifier for a Module and then to get a verifier for a service proxy.
But it is going to be the same verifier!!! So I've decided to implement
a CachingProxyTrust that 1) can be trusted by the client without the
need to issue any remote call (so we need a CachingProxyTrustVerifier
configured on the client)
2) Will cache a verifier it obtains from its delegate
The problem with CachingProxyTrust is that it has to be used by:
a) a Module
b) a service proxy
That's why we need OverrideProxy - it is returned from ModuleExporter to
the service so that the service is unaware of the CachingProxyTrust but
still can use it to obtain its verifier. To trust OverrideProxy the
client has to have OverrideProxyVerifier configured locally.
I hope I clarified everything a little bit...
Michal
