Hi Greg, Thanks so much for your detailed explanation.
So as someone poking around in river for the first time I can say I am pleased with the responsiveness of this list, but the getting started tutorial and the documentation has been difficult & almost non-existent respectively. I do think that you are right that the getting started page should be renovated, and I think that this explanation probably ought to be added somewhere in your docs too. I read some of the dev list archives and noted the worries about the vitality of the community. Ease of adoption is a key factor in growing the community... only users become contributors. You will never know if 100 people try it out and conclude it doesn't work because the getting started tripped them up. Those people often silently come and go. I tend to be a tenacious sort if I see something that looks like it would be cool at in the end of the journey, but not everyone is. Some people are more jaded and less hopeful. They need a quick good first experience to get them started. Those folks may be smarter than I, as sometimes my tenacity makes my progress really slow (but then *sometimes* it allows me to accomplish what others can't or won't). The only projects that can survive difficult documentation are ones that are so cool and fundamental nobody can do without them. About the only one that comes to mind quickly in that category is JUnit, although I've noted that their docs have improved with time, perhaps due to the rise of TestNG? On the flip side are things like Gradle, Hibernate, Ant, Spring, Tomcat, etc which all have copious and usually helpful documentation. Writing docs is terribly boring, but it's how you get people to use your code :) I wonder if you are aware that 3 out of 4 links on your resources page are dead? two give server not found, and a third gives: XML Parsing Error: undefined entity Location: http://jan.newmarch.name/java/jini/tutorial/Jini.xml Line Number 1225, Column 25: This file is Copyright (©) 1999, 2000, 2001, 2003, 2004, 2005 ------------------------^ If I make it over the initial hurdles and feel like I know enough, perhaps I'll contribute some patches to the docs. -Gus On Tue, Jul 8, 2014 at 12:46 PM, Greg Trasuk <[email protected]> wrote: > > Hi Gus: > > Sorry for the delayed reply - I’m on vacation and my internet access is a > little spotty. > > PreferredClassLoader is important in the long run, but you probably won’t > notice the lack of it until you get to some more complicated service > scenarios. In the short term, you could probably do without, however It > might be a little complicated to get rid of. > > Unfortunately, to explain further, we need some more background info. > Bear with me for a few minutes, we’ll come back to the problem, I promise…. > > Jini is a “mobile-code” style of network architecture, although that may > not be entirely obvious. When you pass arguments to a service call, or > receive return values, what’s actually being passed is a serialized, or > “marshalled” version of the object as it exists inside the virtual machine > (VM). This includes “looking up a service” - what you’re receiving is a > marshalled version of the proxy to the that service. > > Serialization is typically performed by ‘net.jini.io.MarshallInputStream’ > and ‘net.jini.io.MarshallOutputStream’. These classes extend > ObjectInputStream and ObjectOutputStream, respectively. The marshalled > form includes a copy of all the data inside the instance, plus the name of > the class, and a “codebase annotation” that provides a url that points to > where the class can be downloaded from (hence mobile code). > > When a marshalled object is loaded into a local virtual machine, the > MarshallInputStream needs to create an instance of the object’s class, and > then put the instance data into it. It will create a new class loader that > loads bytecode from the “codebase annotation url”, and then use that class > loader to create the local instance of the marshalled object. > > Keep in mind that “Foo” loaded by class loader CL-B can implement an > interface “Bar”, or extend a class “Bas” that is loaded by class loader > CL-A. So it’s entirely possible that the user of “Foo” has no knowledge > of, or access to the actual implementation class (e.g. by declaring a > reference to “Bar” or “Bas” and then storing a reference to “Foo” in it). > Also, this applies to fields of an instance as well. > > But what happens if the same class name exists in the local VM’s class > path? In most cases, we’d rather use the locally-defined byte code. > Partly for efficiency reasons (avoid downloading the jar file) but also > because if we want to “use” the class locally, it needs to come from the > same class loader (in other words, “Foo” loaded by class loader CL-A is > actually a different class than “Foo” loaded by class loader CL-B, even if > the actual byte code is the same). So the default behaviour is to use a > locally-available class if there is one, and only use a different class > loader if the required class is not available locally. > > If we load a marshalled object, and we already have the same class > available locally, what happens to the codebase annotation that was on the > marshalled object? It’s gone, since the codebase annotation goes with the > codebase, which clearly is associated with the class loader. Usually > that’s OK, but what if the service really wants to use the code that it’s > providing remotely? That’s where PreferredClassLoader comes in. > > PreferredClassLoader takes a look at a file in the first “jar” file in the > codebase annotation (META-INF/Prefered.list), and treats it as a list of > “Preferred” classes. These are classes where the downloaded code is > “Preferred” to the locally-available class. That way the codebase > annotation sticks, not to mention that the service provides the byte code > that it’s expecting. > > That’s not a really common use case, so as I say, you might not notice the > lack of PreferredClassLoader. It’s plugged in using an SPI provider > (META-INF.services/java.rmi.server.RMIClassLoaderSpi) file that’s in > jsk-resources.jar. So to remove PreferredClassLoader properly, you need to > put a different version of jsk-resources.jar into your class path. > > There’s another possible problem, although I can’t prove it - just a bad > feeling that I get. Having read through all the above, you might notice > that there’s a lot of class loader hocus-pocus going on. Remote code, and > by extension, Jini, is pretty fussy about class loading, and class loader > hierarchy. In addition to on-demand loading (mobile code), the library > uses things like the Preferred list, and Spi configuration files, that are > also loaded through the class loaders. Although there’s nothing illegal, > it’s an advanced use of the class loading mechanism, so it wouldn’t shock > me if the class loader used in your “one-jar” implementation isn’t complete > enough to support it. I’ve come across other class loader implementations > (e.g. Apache Virtual File System) that didn’t work properly, and you > mentioned that your ‘one-jar’ loader has a known bug around loading classes > by name (which, if you think about it, is what needs to happen for just > about all the remote class loading). Also, you need to be able to set the > codebase annotation on a given class loader. > > A “one-jar” library is really a custom class loader that gets it’s byte > code from files that are inside the jar file. So, it’s plausible that any > given “one-jar” approach could cause problems. > > That’s why a container approach is usually better. There’s a rudimentary > container provided with River, ‘com.sun.jini.start.ServiceStarter' which is > unfortunately not used in the “getting started” doc (not sure who wrote it, > but I’m afraid that I personally don’t have time to re-write it just now). > You can see an example of its use in the service registrar startup that’s > mentioned in that doc. More advanced containers are available in the > River-Container project (https://github.com/trasukg/river-container) or > the Rio project (http://rio-project.org). > > Essentially, Jini needs class loaders that are designed for Jini. > (Really, it’s not Jini - mobile code needs mobile-code-aware class > loaders). My experience has been that it’s possible to put simple service > consumers into non-Jini-aware class loaders (e.g. web application > containers like Tomcat, or command-line class paths) with a few > limitations. Hosting service implementations without ServiceStarter, > River-Container, Rio, or some other Jini-aware environment will be a > complete exercise in frustration. > > So, have a look at the way the JavaSpaces implementation gets started up > (it’s the closes to a plain service implementation) and copy that. Or have > a look at River-Container. > > And I suppose we ought to renovate the “getting started” page. You might > also want to have a look at the “hello” example, as mentioned in > http://river.apache.org/doc/info-index.html#example. > > Cheers, > > Greg. > > > On Jul 7, 2014, at 8:19 PM, Gus Heck <[email protected]> wrote: > > > Sorry, sometimes I have a habit of over explaining and obscuring the > > important question... How critical is PreferredClassProvider? Is it > really > > needed? Can I do without it? > > > > > > On Mon, Jul 7, 2014 at 10:23 AM, Gus Heck <[email protected]> wrote: > > > >> Yes the build tool is gradle. It pulls resources from maven central just > >> like maven. It's worth checking out if you haven't. It uses the cool dep > >> resolution stuff from maven but ditches the pom straight-jacket. > >> > >> I found the class, in the jar, overlooked it the first time. I searched > on > >> loader when I opened the jar with emacs and didn't notice that the > package > >> I found was com/sun/jini not net/jini (oops!). So that's resolved, but > this > >> only makes things muddier. The jar is being packed with one-jar (which > is a > >> tool that allows you to produce a jar containing both your classes and > >> their dependencies in the same jar). Clearly many other classes are > loading > >> and so I turned on one-jar's logging and got this: > >> > >> guss-mbp:ingest gus$ j7a -Done-jar.info=true -jar > >> build/libs/ingest-node.jar foo bar 2>&1 | grep Preferred > >> [JarClassLoader] WARN: > >> com/sun/jini/loader/pref/internal/PreferredResources.class in > >> lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > net/jini/loader/pref/PreferredClassLoader$1.class > >> in lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > net/jini/loader/pref/PreferredClassLoader$2.class > >> in lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > net/jini/loader/pref/PreferredClassLoader$3.class > >> in lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > net/jini/loader/pref/PreferredClassLoader$4.class > >> in lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: net/jini/loader/pref/PreferredClassLoader.class > in > >> lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > >> net/jini/loader/pref/PreferredClassProvider$1.class in > >> lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > >> net/jini/loader/pref/PreferredClassProvider$2.class in > >> lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > >> net/jini/loader/pref/PreferredClassProvider$3.class in > >> lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > >> net/jini/loader/pref/PreferredClassProvider$4.class in > >> lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > >> net/jini/loader/pref/PreferredClassProvider$5.class in > >> lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > >> net/jini/loader/pref/PreferredClassProvider$LoaderEntry.class in > >> lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > >> net/jini/loader/pref/PreferredClassProvider$LoaderKey.class in > >> lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > net/jini/loader/pref/PreferredClassProvider.class > >> in lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> [JarClassLoader] WARN: > >> net/jini/loader/pref/PreferredFactoryClassLoader.class in > >> lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with > >> different bytecode) > >> java.lang.NoClassDefFoundError: > net.jini.loader.pref.PreferredClassProvider > >> > >> So I took the jini-ext jar out, and all I get is > >> uss-mbp:ingest gus$ j7a -Done-jar.info=true -jar > >> build/libs/ingest-node.jar foo bar 2>&1 | grep Preferred > >> java.lang.NoClassDefFoundError: > net.jini.loader.pref.PreferredClassProvider > >> > >> So it's there, it's found but something odd is going on after that that > >> results in NoClassDefFoundError. This may relate to the JarClassLoader > that > >> gets installed, but clearly that works most of the time... > >> > >> Are you attempting to manually load this class by looking up the .class > as > >> file? (one-jar has a known bug along these lines. It loads classes just > >> fine from the included jars, and files from your project just fine. I > think > >> it loads files from dep jars just fine too (otherwise lots of stuff > would > >> have issues) but it seems to confused if you try to load classes as if > they > >> are files: https://sourceforge.net/p/one-jar/bugs/73/) > >> > >> How critical is PreferredClassProvider to river's operation? > >> > >> -Gus > >> > >> > >> On Mon, Jul 7, 2014 at 9:39 AM, Greg Trasuk <[email protected]> > >> wrote: > >> > >>> > >>> PreferredClassProvider should be in jsk-platform.jar (which I think you > >>> confirmed with your ‘grep -r’). It is there in the 2.2.2 build that I > >>> have. I’m not familiar with your build tool (Gradle?) but you might > want > >>> to check the class path entry in the jar manifest that’s generated, to > see > >>> if jsk-platform is actually in the runtime class path. > >>> > >>> Are you starting up the system using the ServiceStarter approach or > >>> winging it yourself? If ServiceStarter, could you post the starter > >>> configuration file? > >>> > >>> Cheers, > >>> > >>> Greg Trasuk. > >>> On Jul 6, 2014, at 8:27 PM, Gus Heck <[email protected]> wrote: > >>> > >>>> I've now spent several hours hunting for this class. What jar contains > >>> it? > >>>> > >>>> Presently I'm building with the following (shotgun, messy non > minimized) > >>>> dependencies: > >>>> dependencies { > >>>> compile 'com.google.guava:guava:15.0' > >>>> compile 'com.google.code.findbugs:jsr305:2.0.3' > >>>> compile 'jini:jini-core:2.1' > >>>> compile 'org.apache.river:reggie:2.2.2' > >>>> compile 'net.jini:jini-ext:2.1' > >>>> compile 'net.jini:jsk-lib:2.2.2' > >>>> compile 'net.jini:jsk-dl:2.2.2' > >>>> compile 'net.jini:jsk-platform:2.2.2' > >>>> compile 'net.jini:jsk-resources:2.2.2' > >>>> compile 'net.jini:jini-core:2.1' > >>>> compile 'net.jini:jini-ext:2.1' > >>>> compile 'net.jini:jsk-policy:2.2.2' > >>>> // compile 'net.jini:jini-starterkit:2.1-beta2' // 404 in maven > >>> central > >>>> compile 'org.apache.logging.log4j:log4j-core:2.0-rc2' > >>>> testCompile 'junit:junit:4.11' > >>>> } > >>>> > >>>> I basically got frustrated and just threw everything I could find at > it > >>> and > >>>> it still does this: > >>>> > >>>> guss-mbp:ingest gus$ alias j7a > >>>> alias > >>>> > >>> > j7a='/Library/Java/JavaVirtualMachines/jdk1.7.0_60.jdk/Contents/Home/jre/bin/java > >>>> -Djava.security.policy=/Users/gus/tools/jpolicy/all.policy > >>>> > >>> > -Djava.rmi.server.RMIClassLoaderSpi=net.jini.loader.pref.PreferredClassProvider' > >>>> guss-mbp:ingest gus$ j7a -Done-jar.silent=true -jar > >>>> build/libs/ingest-node.jar foo bar > >>>> Starting injester node... > >>>> Jul 06, 2014 7:57:31 PM > >>>> net.jini.discovery.LookupDiscovery$UnicastDiscoveryTask run > >>>> INFO: exception occurred during unicast discovery to > guss-mbp.lan:54290 > >>>> with constraints InvocationConstraints[reqs: {}, prefs: {}] > >>>> java.lang.NoClassDefFoundError: > >>> net.jini.loader.pref.PreferredClassProvider > >>>> at > >>>> > >>> > java.rmi.server.RMIClassLoader.initializeProvider(RMIClassLoader.java:687) > >>>> at java.rmi.server.RMIClassLoader.access$000(RMIClassLoader.java:110) > >>>> at java.rmi.server.RMIClassLoader$1.run(RMIClassLoader.java:120) > >>>> at java.rmi.server.RMIClassLoader$1.run(RMIClassLoader.java:119) > >>>> at java.security.AccessController.doPrivileged(Native Method) > >>>> at java.rmi.server.RMIClassLoader.<clinit>(RMIClassLoader.java:117) > >>>> at net.jini.loader.ClassLoading.loadClass(ClassLoading.java:138) > >>>> at > >>>> > net.jini.io.MarshalInputStream.resolveClass(MarshalInputStream.java:296) > >>>> at > >>>> > java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1612) > >>>> at > >>> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517) > >>>> at > >>>> > >>> > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771) > >>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350) > >>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370) > >>>> at net.jini.io.MarshalledInstance.get(MarshalledInstance.java:358) > >>>> at > >>>> > >>> > com.sun.jini.discovery.DiscoveryV1.doUnicastDiscovery(DiscoveryV1.java:397) > >>>> at > >>> net.jini.discovery.LookupDiscovery$13.run(LookupDiscovery.java:3327) > >>>> at java.security.AccessController.doPrivileged(Native Method) > >>>> at > >>>> > >>> > net.jini.discovery.LookupDiscovery.doUnicastDiscovery(LookupDiscovery.java:3324) > >>>> at > >>>> > >>> > net.jini.discovery.LookupDiscovery.doUnicastDiscovery(LookupDiscovery.java:3355) > >>>> at > >>>> > net.jini.discovery.LookupDiscovery.access$3900(LookupDiscovery.java:696) > >>>> at > >>>> > >>> > net.jini.discovery.LookupDiscovery$UnicastDiscoveryTask.run(LookupDiscovery.java:1744) > >>>> at > >>> com.sun.jini.thread.TaskManager$TaskThread.run(TaskManager.java:331) > >>>> > >>>> No Service Registries found > >>>> > >>>> I'm passing in the option for this class on the advice of your getting > >>>> started page < > >>> http://river.apache.org/user-guide-basic-river-services.html>. > >>>> It would be really nice if that page listed what dependency I needed > for > >>>> this. > >>>> > >>>> I tried grepping the entire distribution and only found it in two > places > >>>> > >>>> guss-mbp:apache-river-2.2.2 gus$ grep -r PreferredClassProvider * > >>>> <*snip* javadoc matches> > >>>> Binary file lib/jini-ext.jar matches > >>>> Binary file lib/jsk-platform.jar matches > >>>> > >>>> in both jars I see only: > >>>> drwxr-xr-x 0 10-Nov-2013 22:49:20 com/sun/jini/loader/ > >>>> drwxr-xr-x 0 10-Nov-2013 22:49:20 com/sun/jini/loader/pref/ > >>>> drwxr-xr-x 0 10-Nov-2013 22:49:20 > >>>> com/sun/jini/loader/pref/internal/ > >>>> -rw-r--r-- 7218 10-Nov-2013 22:49:20 > >>>> com/sun/jini/loader/pref/internal/PreferredResources.class > >>>> > >>>> > >>>> But in any case as you can see both of these libs are already in my > >>>> dependency list. > >>>> > >>>> This puzzles me since it's clearly in your repository... > >>>> > http://svn.apache.org/viewvc/river/jtsk/trunk/src/net/jini/loader/pref/ > >>>> > >>>> shouldn't it be in at least one of the distributed jars, if not one of > >>> the > >>>> packages on mavencentral? > >>>> > >>>> -Gus > >>>> > >>>> -- > >>>> http://www.the111shift.com > >>> > >>> > >> > >> > >> -- > >> http://www.the111shift.com > >> > > > > > > > > -- > > http://www.the111shift.com > > -- http://www.the111shift.com
