So, Just a quick update.
I gave the client-side ClassLoader encapsulation idea some more thought and rejeted it (sorry) for the reasons that : - I've lived through J2EE ClassLoader hell and don't want to repeat the experience - I'm looking towards an architecture where I can define new types on any node of my Clojure data-grid and have them accessible in any other node. This might be done by keeping my class-name:bytecode table in a clustered cache and inserting a single ClusteredClassLoader (NYI) into the client's ClassLoader hierarchy. Each client would be able to mix classes from multiple server JVMs without having to be concerned about keeping them all segregated. So, I introduced the concept of a per-jvm id and hacked it into RT, Compiler and LispReader. There were not too many places that needed to be changed. The result is, I can now create a type in one jvm, create an instance of the type, serialise it, ship the bytes to another jvm and deserialise it, at which point the instance's class is pulled from the original jvm. No more nasty exceptions because of name collisions etc. I have plumbed this infrastructure in underneath my app and, with the exception of some unrelated bugs that I am chasing down, everything seems to work beautifully :-) There are some limitations with this technique that I haven't looked into too deeply yet - the most obvious one is that whilst remote types can be made available within a local jvm, this is only at the java level, so extra things that defrecord might do for you like intern the classname as a symbol in your current namespace are not being done - which, I can currently live with, but I can see this being a tripping point to a completely transparent solution. For me, this is a great leap forward as I have found each Clojure JVM an island until I got this working, but now I should be able to build a distributed app in Clojure without having to sacrifice Clojure's dynamicity in the process - I can have my cake and eat it :-) I'm going to play with it for a few days, iron out all my other issues and investigate exactly what I have achieved, then I will post back a proper description in case anyone else is trying to build distributed Clojure apps in a similar manner. cheers Jules On Mar 3, 2:30 pm, Jules <jules.gosn...@gmail.com> wrote: > On Mar 3, 1:22 pm, Alessio Stalla <alessiosta...@gmail.com> wrote: > > > > > > > On Thursday, March 3, 2011 11:46:03 AM UTC+1, Jules wrote: > > > > Thanks, Alessio, > > > > I did know this, but it is a welcome addition to the thread. > > > Ok. Classloaders are a tricky matter and many people don't have clear ideas > > about them - sorry for assuming you were one of those people :) > > np :-) > > > > I reread my last posting and it was a bit confused - I've done a > > > little research in the Compiler class and can now clarify what I think > > > is happening. > > > > Clojure 1.3.0-alpha4 > > > user=> (type (fn [])) > > > user$eval1$fn__2 > > > user=> > > > > Here I have created a new function and asked it for its class. Clojure > > > has emitted bytecode on the fly to implement this and packed this into > > > a Java Class. A Class has a name unique - as you pointed out - within > > > its ClassLoader, so to avoid name collisions, Clojure attempts to > > > create a unique name for the class. This is composed, amongst other > > > things from its namespace and RT.nextID() (I'm assuming RT = RunTime). > > > > RT.nextID seems to start at 0 and allocate ids sequentially - lets > > > create another fn : > > > > user=> (type (fn [])) > > > user$eval5$fn__6 > > > user=> > > > > I suspect that the gap between 2 and 6 is due to other processes going > > > on behind the scenes which require ids, rather than some intentional > > > design - but haven't checked. > > > > This all works fine in a single JVM - no two functions will end up > > > creating types with the same name - however if I take the class for a > > > function from [a ClassLoader in] one clojure runtime and copy it > > > across to another clojure runtime that has already allocated this > > > classes id to another [in the receiving ClassLoader] then I would see > > > a collision. > > > > I think that this is what is happening above. I am serialising an > > > object in one jvm, putting it into a message and sending the message > > > to another jvm. Upon receipt I attempt to deserialise the object > > > contained in the message. This involves looking up its class by name > > > [within the scope of a ClassLoader] - but the name has already been > > > allocated to a local class [within the same ClassLoader] which is > > > mistakenly then used to try to deserialise the object. The object's > > > data is not presented in the format that the local class expects for > > > one of its own instances - hence the Exception in my posting above. > > > > This is currently only theory. > > > Makes sense. > > > > I think that the easiest way to fix it is to use not just a jvm-local > > > uid as part of the class name, but also an id which uniquely > > > identifies this jvm amongst all the jvms involved in my cluster. It > > > looks like not too many classes make use of RT.nextID(), so I think my > > > next move will be to add an e.g. RT.jvmID and use this in conjunction > > > with RT.nextID(). If this resolves my problem then there is a good > > > chance that my supposition was correct - otherwise it is back to the > > > drawing board. > > > I think a better option is to make sure that the classloader you use for > > deserializing is completely separated from the classloader Clojure uses to > > load compiled functions. Then, no possibility of collision exists, > > regardless of how you generate class names.- Hide quoted text - > > > - Show quoted text - > > That's an interesting idea, that I haven't really pursued, I think, > OTOH, because client-side, I think that I want to be able to mix > objects that herald from both local and remote jvms... - but this is > just a gut feeling, I haven't really looked for any concrete usecases > in any detail, so I'll give your suggestion consideration and see > whether it would fit the bill. > > Using your approach would also be a good way of checking that my > assumption about the name collision was correct, so I may see if I can > set up my code to use a different ClassLoader in which to deserialise > to see if that solves my problem. > > Thanks for this idea, it may help me a lot :-) > > Jules- Hide quoted text - > > - Show quoted text - -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
