Thanks Gregg, much appreciated.
Gregg Wonderly wrote:
Getting back around to some of these, hopefully no confusion...
Yea, can be difficult to express things using only text communication,
especially with something so complex. Replies inline below.
Peter Firmstone wrote:
Hi Gregg,
Sorry I missed this earlier, have a look at my latest messages, they
might provide some more background to my thoughts.
Mike Warres has a good example of when the existing mechanism causes
codebase annotation loss and type conflicts on page 23 of his paper
"Class Loading Issues in Java RMI and Jini Network Technology".
https://issues.apache.org/jira/secure/attachment/12413650/Java+Classloader+issues+relating+to+Jini+smli_tr-2006-149.pdf
Because the Service API is loaded from the local file system and
isn't dynamically downloaded, it may not have an annotation, I'm
thinking about creating a new URL scheme, that provides information
about all Jar files, whether local or downloaded dynamically, such as
jar name and version annotation. This should help people who wish to
provision their codebases.
For me, the important thing to understand about class
resolution/loading, is that there are two distinct issues. First, do
you have the correct version of the class somewhere in the class
loading system in use, and Second, if the right version or no version
is available, do you have a way to go look for a definition that would
be the right version to use?
ClassLoaders have historically been the domain of issue #1. Issue #2
has been dealt with in a surprisingly large number of ways.
Serialization annotations that MarshalledObject et.al. provide one way
to deal with #2. But we also have later development of things like
OSGi, Maven and other packaging systems which include ways to declare
or facilitate resolution of missing dependencies.
RMIClassLoaderSPI is the pluggable way to deal with the String
annotations on classes. It can facilitate a wide number of
possibilities. The predominate issues is that it doesn't include an
"implementation" indication in concert with the String value, so you
don't know how to "delegate" to other mechanisms to make use of that
String value in a "varied" way.
It seems to me that if we do something with a new RMIClassLoaderSPI
implementation. My changes add no real value other than facilitating
the addition of another implementation that is offered remotely or
through some other path. What we really need is the ability to look
at the annotation and use it intelligently to ask a factory mechanism
to use it to create a class loader.
Note I haven't had any thoughts about removing
PreferredClassProvider, but I'm thinking about another ClassLoader
structure, although I'm prone to changing my mind as I struggle to
understand it all. I haven't got any implementations, yet, still
working it all out.
PreferredClassLoading allows you to fix code that is generally used
across an application "suite", by overriding the use of something in
the classpath with something you provide in your -dl.jar (or other
codebase content) by making it preferred.
Ok, this is something I'm trying to fix using a different approach with
Classloaders. If the Application code is not visible to the proxy, the
proxy can have it's own independent classes, with no class sharing
between the proxy and application other than platform and API classes.
I'd like a way to define classes for loading into an application or
service classloader too, similar to a classpath. Smart Proxy's can
utilise their own libraries without conflicting with the application's
library versions.
This is sort of what I'm thinking, evolved since last post, based on
comments:
CLASS LOADER VISIBILITY
_______________________________________________________________________
| |
| Platform & API ClassLoader (incl Dumb Proxy Objects) |
|----------------|------------------------------------------------------|
| _________|________________________________ |
| | | | | |
| ServiceUI | Smart Proxy Extended Service API |
| ClassLoader | (incl ServiceUI) & Dumb Proxy ClassLoader |
| (UI that uses | ClassLoader's | |
| Dumb Proxy's) | ___________________|___________ |
| |_________ | | | |
| | | | ServiceUI Smart Proxy |
| | Parameter Imp ClassLoader's ClassLoader's |
| | ClassLoader's (for Dumb Proxy's) |
| | (Server side) |
|----------------|------------------------------------------------------|
| | |
| Common Classloader (As Per Dennis' comments) |
| _________|________________________ |
| | | |
| Service Imp Application |
| ClassLoader's ClassLoader |
| (incl Parameter Imp classes) |
|_______________________________________________________________________|
Everything between the two --- lines is downloaded dynamically, cached
or provisioned.
Dumb Proxy's have no download archives or classes other than those in
the API,
so can be safely loaded into the Platform API ClassLoader or Extended
Service API
ClassLoader. These classloaders will be available via a weak reference
index.
All classes in child ClassLoaders can see all parent ClassLoaders above
in the tree, but classes in parent ClassLoaders cannot see the classes
below them in child ClassLoaders. For example, classes in the Platform
& API CL, Common CL are visible to the Application CL classes.
This visibility, between ClassLoader's mapped out above, prevents
implementations from interfering with each other, they can only
cooperate between each other using common API classes. No more worries
about versioning conflicts etc. Versions can be specified by
implementations to allow codebase caching and provisioning to ensure
that Serialization and implementation compatibility remains between the
Server and it's Proxy.
This may cause some libraries to become duplicated in memory, however we
must ensure a particular version is only downloaded once.
Something that has my attention about security is, currently we've based
proxy security on the ClassLoader. However when the AccessController
checks permission, it allows only the permissions common to all
ProtectionDomains on the Stack (see AccessControlContext). Permissions
really should be dynamically granted to ProtectionDomains, not the
ClassLoader.
A ProtectionDomain, might represent a Principal, or it might represent a
downloaded codebase, any number of which may exist in a single
ClassLoader (so long as classes are not duplicated, this should be
avoided since duplicate classes cannot be loaded into a ClassLoader).
Hence if a number of Services use the same codebases for their smart
proxy's then that codebases ProtectionDomain will need the required
Permissions and so will the ProtectionDomain for a Principal in order
for those Permissions to become effective. So we can preserve security
and have multiple Services share the same proxy code. But to do so we
must forget about ClassLoader based Permission grants.
We can base Codebase trust on:
1. Certificates[] "Who wrote it?"
2. CodeSource "Who wrote it and the name of the Codebase?"
We can base Principal trust on:
1. Principals[] "Who are you?"
When a ProtectionDomain is created with the constructor:
public ProtectionDomain(CodeSource codesource,
PermissionCollection permissions,
ClassLoader classloader,
Principal[] principals)
The permissions granted to the ProtectionDomain will be dynamic and the
current Policy will always be asked prior to the PermissionCollection
passed into the constructor (Performance Hint: make sure permissions is
null, use the Policy only)
The Principal[] can be java.security.acl.Group[] objects (Group extends
Principal), then your free to add and remove Principals to that Group
and hence the Group's ProtectionDomain, that associated with the
CodeSource of the Proxy. A group might represent each type of Service.
ProtectionDomains representing each will be on the stack and only the
Permissions common to all ProtectionDomains on the stack will be granted.
Some observations:
1. Codebases should advertise their required Permissions if trust is
required.
2. Codebases should be signed if trust is required.
3. Utility Library Codebases can be released and signed for others to
"trust" to be utilised in proxy's. You might not trust unknown
proxy code, but you might trust the utility library that provides
trusted functionality for it.
So client Proxy trust is currently:
1. Download the Proxy codebase.
2. Unmarshall the Proxy.
3. Ask the Proxy for the Proxy verifier.
4. Ask the Server if it trusts the proxy.
5. Grant trust to the proxy ClassLoader.
We can optionally do this too instead (feel free to correct any poorly
conceived assumptions):
1. Download the Proxy codebase (unless cached or downloaded prior)
2. Record the Permissions required by the CodeSource advertised in
the jar file.
3. If Permissions are required, Create a Group and Create a
ProtectionDomain for that CodeSource, passing the new Group
(Principal) to the constructor (on first use of the codebase). A
Group might be specific a particular Service Interface and may
already exist.
4. Unmarshall the Proxy
5. Check Constraints.
6. Authenticate the Service if required.
7. Authenticate the Client if required.
8. Ask the Proxy for the Proxy Verifier.
9. Does the Server trust the Proxy?
10. Add the Service's Principal (or it's Group) to the Group, in the
ProtectionDomain.
11. Grant trust
1. Using grantCodeSource(CodeSource cs, Principal[] groups,
Permission[] permissions)
2. OR grantProtectionDomain(Class cl, Permission[] permissions)
3. OR grant(Class cl, Principal[] principals, Permission[]
permissions) - ClassLoader grant.
4. OR grant(Certificate[] certs, Principal[] principals,
Permission[] permissions)
Notes:
* The permissions granted are limited to those declared in the jar file.
* If the Trust grant is performed using 11.3. then permission grants
are the same as the current ClassLoader based behaviour, all other
libraries in with the proxy will be loaded into that ClassLoader
and also receive those permissions.
* Permissions granted using 11.4 are granted based on the codebase
signers (developers) and Principals (groups) and may be granted at
any time prior.
* Permissions granted using 11.2 are granted directly to the proxy
ProtectionDomain and only apply to the proxy's Code source, not
bundled libraries or other jar archives distributed with the proxy.
* Permissions granted using 11.1 are granted directly to the
CodeSource and Principals (groups) and may be applied prior to
downloading the code source.
* Other proxy objects from different services, with identical proxy
CodeSource may share the ProtectionDomain and class files, in this
case the trust sequence is limited to steps 4 to 10 as the
CodeSource itself is already trusted, the only thing that remains
is to verify the proxy and perform authentication if required.
These Dynamic Grants have been implemented in
DynamicConcurrentPolicyProvider. Permissions granted to Groups differ
from current Policy implementations that test Principals for equality
only, this new Policy implementation also implies true when Principals
are members of Groups that have been granted Permissions.
It might help to provide a Principal and Group framework implementation
to make Authentication easier.
So this does actually point to a use for Patrick's suggested Codebase
Entry? Yes, for example if many services existed on the internet that
utilise the same codebase versions, then a client could specify only a
particular version, to ensure it only downloads one codebase, once for
all matching instances of a Service. It might first get the Codebase
Entry's for a particular Service type, then lookup by each codebase
version and deal with them in compatible batches. Waiting for the
Codebase URL annotation would be too late, filtering would be performed
at the client rather than by the ServiceRegistrar.
We do need to implement something like RMIClassLoaderSPI, where
ClassLoaders are created based on the names of jar archives *proxy.jar,
*ext-api.jar, *ui.jar and *param.jar. Where versioning is taken into
consideration and a new URL format is created that allows for the needed
flexibility. Then of course class resolution needs to be figured out
with the new URL annotation.
This may take some time to digest. Please see the code in:
https://svn.apache.org/repos/asf/incubator/river/jtsk/trunk/src/org/apache/river/
Cheers,
Peter.
I find this mechanism extremely useful in my service UI client that
has the same classes as all my UIs use for Swing and AWT stuff. If I
need additional functionality or to fix something that only the UI of
a service needs, then I can make that class preferred and not have to
update all of my service UI client instances. Mobile code solutions
are very convenient.
Gregg Wonderly
Gregg Wonderly wrote:
I have a Jini based application that is a content based router
system for satellite networks. There are multiple servers running
with comms cards, and the ServiceUI needs to see all of them at
once, because I use transactions to commit data changes to all
servers. In this case, one clients ServiceUI must lookup all of the
services, ignoring itself, and then get the service proxies to work
with. All of the services proxies need to share interfaces and data
classes that are commonly exchanged.
I didn't need a different kind of classloader tree, I just needed to
make sure that the parent class loader of the RMIClassLoaderSPI was
the ClassLoader of the client (The context ClassLoader which is a
PreferredClassLoader), which was already happening. All the other
service instance unmarshalling would in fact make use of the
PreferredClassLoader, so that versioning could be done by preferring
classes, for example, and each services preferred classes would be
honored.
There is only one -dl.jar involved in the commonality aspects of the
involved Classes. I could break the common classes into a separate
jar, but I have not done that, yet.
This type of application must be restarted fairly rarely, so class
compatibility is a very key issue.
Peter, can you provide a more specific example of when you think the
structure you are proposing would be valuable, i.e. the existing
mechanism would break?
Gregg Wonderly
