Hi Gregg,
I hear what your saying.
Can I share some thoughts?
An Object's Class Type is the fully qualified class name + the originating
ClassLoader.
HTTP codebase's are part of the problem, the URLClassLoader is fixed in the
object's Type (class identity), which may change over time. Michael Warres
addressed this problem by creating a dynamic codebase service where the URL was
a cryptographic hash of the jar file (stored data) identity.
Michael made a presentation on Service based codebases, apparently not
much code was required to implement it. We cannot directly copy the
code (interfaces etc) from the presentation due to copyright, although
we can produce functionally equivalent code.
http://www.jini.org/files/meetings/eighth/presentations/Warres/Warres.pdf
See also River-316
So the dynamic service based codebase could move around, and be offered
redundantly.
In addition, we could update/upgrade/replace the Hierarchical based
PreferredClassLoader relationship with a more flexible DynamicClassLoader based
on ClassWorlds to segregate incompatible class packages while granting
compatible classes the ability to communicate. There is BSD licensed code that
we could build on:
http://classworlds.codehaus.org/ ClassWorlds has some very elegant simple models
(easy to code & use) that may help us.
From the above website: "The |classworlds| model does away with the hierarchy normally
associated with ClassLoaders. Instead, there is a pool of ClassRealms
<http://classworlds.codehaus.org/apidocs/com/codehaus/classworlds/ClassRealm.html> which
can import arbitrary packages from other ClassRealms. Effectively, |classworlds| turns the
old-style hierarchy into a directed graph."
One might give a library its own ClassLoader in each JVM for instance,
then we could make that library available to the applications / services
that depended upon it. A later version of that library would have a
separate ClassLoader so that "Jar hell" or "Classpath Hell" (standalone
JVM talk) or its distributed equivalent "Class Type Hell" or
"ClassLoader Hell" are avoided (unfair: ClassLoaders are a blessing in
disguise, they add another dimension to namespaces).
The new com.sun.jini.tool.classdepend package has functionality to
record class dependency relationships in an array, this could be
modified to also store each classes unique sha-1 or md5 hash, as well as
serialVersionUID's, if they implement Serialisable or Externalizable,
this dependency array or a similar string hash (similar to your
suggestion on namespaces) could be returned on request as part of the
codebase service.
I purchased a copy of the following paper online (Thank's Jim for the
tip), I found a freely available copy you can all read. Its called
Modular Software Upgrades for Distributed Sytems by Sameer Ajmant,
Barbara Liskov and Liuba Shrira. It discusses updating services in
distributed systems. It is a course grained versioning system.
http://www.pmg.csail.mit.edu/~ajmani/papers/ecoop06-upgrades.pdf
So I'm currently trying to get my head around a new ClassLoader
framework for classes where a developer knows which objects need to be
preserved over time, while their underlaying class implementations may
change, this would be a fine grained versioning system, at the Class
Level. The new package com.sun.jini.tools.classdepend can return a
dependency tree stored in an array, this could be extended to record
each classes unique sha-1 or md5 hash, as well as serialVersionUID's, if
they implement Serialisable or Externalizable, it already stores the
fully qualified class name. The sha-1 or md5 hash would also form part
of Security of downloaded code as this could be checked before loading
the Class file.
Versioning, Identity and preservation of object state / contents over
time is much harder and is unsolved. Jim Waldo recommends Classes
implement interfaces, where only the interfaces are used. This allows
objects from different ClassLoaders that implement the same interface to
interact and be interchanged. For instance if you have a Class called
Money.class (version 1) and you reimplement its methods and add
additional methods in Money.class(version 2), the only way the objects
can be used in the same array etc is if they share a common Interface or
ancestor class. If you inherit Money.class (version 1) and call it
MoneyTree.class you can override all replaced methods and add additional
methods and it can be used as a money object, however you can't use the
new additional methods when in company with the original class Money's
objects, only those that existed prior.
To overcome this problem, I'm thinking about a ClassLoader Versioning
Framework for Objects we want to distribute and preserve over time,
preserving their state and contents while retaining the ability to
upgrade their class files or bytecodes and also changing their type,
using Interfaces to enable interoperability between objects with
different types. Serialization can be used to upgrade objects class file
bytecodes (marshall, unmarshall into a replacement ClassLoader) with
required visibility granted by what we can build on using ClassWorlds.
Any objects linking to the VersionedObjects could be strongly linked via
a reference object that was updated with the VersionedObject's new hard
reference location, all other objects could be strongly linked to the
ReferenceObject and retrieve a weak link from the reference object to
the VersionedObject. The ClassLoader Versioning Framework would keep a
weak reference to each ReferenceObject with a Lease, after the Lease
expires, the ClassLoader would check if that object still existed
(weakly referenced, may have been garbage collected) and if so, check if
its Class file has been updated via an Update Service, which returns the
hash code for the update Class file. Alternately this could be
requested via a ClassLoader method, each time a weak reference is
requested through the ReferenceObject. The VersioningClassLoader would
then, with the hashcode, request a codebaseURL from the Codebase Service
and create the new version in another ClassLoader, the new
VersionedObject would be strongly referenced by the existing
ReferenceObject, leaving the old VersionedObject with no strong
reference, to be Garbage Collected.
All VersionedObject's must implement Serializable.
A Mutable object would require a Transaction Manager.
Identity is more difficult, for objects who's identity is sufficiently
determined by the equals() and hashCode() methods this should be
sufficient. However objects that require a unique identity can be
broken down again into two types:
1. Immutable objects where Object.equals() doesn't determine object
Identity.
2. Mutable objects where Object.equals() doesn't determine object Identity.
I think #1 could be handled by an ObjectUniqueID service that provides
three unique numbers; The time in milliseconds at the time of request
and two random numbers. The object would receive this service at
instantiation time if required. The likelihood that two random numbers
and the time in milliseconds would produce a match for the same fully
quallified class name would be vary rare indeed.
Well #2 would be more complex, this object would need a Transaction
Manager, and also require the ObjectUniqueID service at the time of
instantiation.
With this in mind, I have no idea how to instantiate or construct a
VersionedObject, I haven't yet figured this part out, perhaps it could
be implemented by:
Interface VersioningClassLoader {
public ReferenceObject instantiate(Builder builderObject, String
fullyQualifiedClassName){};
}
Calling methods on the VersionedObject could be done by:
ReferenceObject ob = versionedClassLoaderInstance.instantiate(builder,
"my.package.classname");
ACommonInterface foo = ob.getWeakRef(); // The ReferenceObject checks
the lease is valid first.
or to execute some method:
ob.getWeakRef().someMethod();
When all strong links to the ReferenceObject and the VersionedObject it
points to, go out of scope, the ReferenceObject and the VersionedObject
can be garbage collected.
Each ClassLoader would be garbage collected after all VersionedObjects
it manages go out of scope (no strong references left) objects whose
leases have expired would be migrated to new ClassLoaders, each
classloader might have a maximum age determined by maximum Lease time +
a time window during which it can be given new VersionedObjects.
All objects that the VersionedObjects depend upon, would be considered
supporting objects and would go out of scope once the VersionedObjects
do, for example a new class file upgrade might dictate another version
of an external library, new objects required by the VersionedObject
would have to be created during unmarshalling and the new class files
for the library downloaded via the codebase Service, the codebase
service would also advertise via the service, the dependency tree it
contains, so the VersioningClassLoader Framework would determine the
Libraries suitability and make it available via a LibraryClassLoader if
it isn't already available.
All objects upon unmarshalling would be checked to see if they currently
exist in memory based on Lease Validity, identity or equality and if so
the in memory object used.
What I'm talking about is conceptual and experimental, I'm hoping others
will be able to provide some thoughts / input, assist and see if we
can't produce something useful along with the changes you've made and
lessons learned. Or alternatively tell me I'm totally nuts ;)
Cheers,
Peter.
Gregg Wonderly wrote:
Niclas Hedhman wrote:
OSGi zealots don't like Jini for many reasons, probably in reality
driven by politics more than technology. One key technology that has
repeatedly been mentioned to me as a show-stopper for even considering
Jini in a more central role, is the differences in "service
attributes/properties". Jini's 'exact-match' Entry matching is
considered inferior of the LDAP expressions of OSGi. Somehow it feels
like a weak argument, and I think River today would consider
convergence on this point.
The issue which I think is never fully considered is that lookup is
based on the Java type system, including complex types, and not based
on string matching. If it was just string matching, we'd have RE
support now. But there is no defined RE syntax for "derived from" or
"implements X" etc. The Java type system provides that.
I will concede that the mechanics of matching in reggie are a
reimplementation of the type system semantics, because code is not
unmarshalled (as a versioning, code corruption, and security measure,
at the least).
I'm more than willing to put together a new lookup service that does
provide "string only" lookup of Entry.toString() values. It would be
possible to include new data, taken from the Entry values before they
are marshalled for transport.
My changes to reggie to support deferred downloading, include the
packaging of all class and interface names as part of the
MarshalledInstance so that you can ask if an Entry (or the service
itself) "is a" without having to unmarshall it.
Mobile code is non-trivial. Many of us have experience now, and River
has some additional tools that were not in place before
(PreferredClassLoader for one) which can "help" manage class
dependency trees so that unmarshalled object can avoid being corrupted.
My http://griddle.dev.java.net project illustrates that you can have a
"Javaspace" like thing with live code, including matching using code.
I separated the keys from the data though so that you can use native
types for keys and unmarshalling them doesn't "load" any code.
There are lots of things that I suspect the OSGi camp is only now
discovering to be necessary "evils". Some will be things we can work
around or do differently.
Designing a completely new lookup, and getting it into the "Jini
world" might be a useful task, but we will only be able to do that one
time, I'm guessing.
OSGi was aimed at "everyone you want, can participate" in this VM.
Jini was aimed at "everyone who wants to, can run code" in this VM.
There are different considerations for each.
Participation in OSGi is selected participation because the container
is loaded with what you want to run. Jini is open participation,
because the network is full of things that want to run. It's the
subtle, but important difference in who initiates the installation of
the software that makes all the difference.
Gregg Wonderly
