Hi, This sounds very interesting and useful.
Is the source code available somewhere? Or a more detailed description? Thanks On Fri, Mar 4, 2011 at 1:48 AM, Jules <jules.gosn...@gmail.com> wrote: > > So, > > Just a quick update. > > I gave the client-side ClassLoader encapsulation idea some more > thought and rejeted it (sorry) for the reasons that : > > - I've lived through J2EE ClassLoader hell and don't want to repeat > the experience > > - I'm looking towards an architecture where I can define new types on > any node of my Clojure data-grid and have them accessible in any other > node. This might be done by keeping my class-name:bytecode table in a > clustered cache and inserting a single ClusteredClassLoader (NYI) into > the client's ClassLoader hierarchy. Each client would be able to mix > classes from multiple server JVMs without having to be concerned about > keeping them all segregated. > > So, I introduced the concept of a per-jvm id and hacked it into RT, > Compiler and LispReader. There were not too many places that needed to > be changed. > > The result is, I can now create a type in one jvm, create an instance > of the type, serialise it, ship the bytes to another jvm and > deserialise it, at which point the instance's class is pulled from the > original jvm. No more nasty exceptions because of name collisions etc. > > I have plumbed this infrastructure in underneath my app and, with the > exception of some unrelated bugs that I am chasing down, everything > seems to work beautifully :-) > > There are some limitations with this technique that I haven't looked > into too deeply yet - the most obvious one is that whilst remote types > can be made available within a local jvm, this is only at the java > level, so extra things that defrecord might do for you like intern the > classname as a symbol in your current namespace are not being done - > which, I can currently live with, but I can see this being a tripping > point to a completely transparent solution. > > For me, this is a great leap forward as I have found each Clojure JVM > an island until I got this working, but now I should be able to build > a distributed app in Clojure without having to sacrifice Clojure's > dynamicity in the process - I can have my cake and eat it :-) > > I'm going to play with it for a few days, iron out all my other issues > and investigate exactly what I have achieved, then I will post back a > proper description in case anyone else is trying to build distributed > Clojure apps in a similar manner. > > cheers > > > Jules > > On Mar 3, 2:30 pm, Jules <jules.gosn...@gmail.com> wrote: >> On Mar 3, 1:22 pm, Alessio Stalla <alessiosta...@gmail.com> wrote: >> >> >> >> >> >> > On Thursday, March 3, 2011 11:46:03 AM UTC+1, Jules wrote: >> >> > > Thanks, Alessio, >> >> > > I did know this, but it is a welcome addition to the thread. >> >> > Ok. Classloaders are a tricky matter and many people don't have clear ideas >> > about them - sorry for assuming you were one of those people :) >> >> np :-) >> >> > > I reread my last posting and it was a bit confused - I've done a >> > > little research in the Compiler class and can now clarify what I think >> > > is happening. >> >> > > Clojure 1.3.0-alpha4 >> > > user=> (type (fn [])) >> > > user$eval1$fn__2 >> > > user=> >> >> > > Here I have created a new function and asked it for its class. Clojure >> > > has emitted bytecode on the fly to implement this and packed this into >> > > a Java Class. A Class has a name unique - as you pointed out - within >> > > its ClassLoader, so to avoid name collisions, Clojure attempts to >> > > create a unique name for the class. This is composed, amongst other >> > > things from its namespace and RT.nextID() (I'm assuming RT = RunTime). >> >> > > RT.nextID seems to start at 0 and allocate ids sequentially - lets >> > > create another fn : >> >> > > user=> (type (fn [])) >> > > user$eval5$fn__6 >> > > user=> >> >> > > I suspect that the gap between 2 and 6 is due to other processes going >> > > on behind the scenes which require ids, rather than some intentional >> > > design - but haven't checked. >> >> > > This all works fine in a single JVM - no two functions will end up >> > > creating types with the same name - however if I take the class for a >> > > function from [a ClassLoader in] one clojure runtime and copy it >> > > across to another clojure runtime that has already allocated this >> > > classes id to another [in the receiving ClassLoader] then I would see >> > > a collision. >> >> > > I think that this is what is happening above. I am serialising an >> > > object in one jvm, putting it into a message and sending the message >> > > to another jvm. Upon receipt I attempt to deserialise the object >> > > contained in the message. This involves looking up its class by name >> > > [within the scope of a ClassLoader] - but the name has already been >> > > allocated to a local class [within the same ClassLoader] which is >> > > mistakenly then used to try to deserialise the object. The object's >> > > data is not presented in the format that the local class expects for >> > > one of its own instances - hence the Exception in my posting above. >> >> > > This is currently only theory. >> >> > Makes sense. >> >> > > I think that the easiest way to fix it is to use not just a jvm-local >> > > uid as part of the class name, but also an id which uniquely >> > > identifies this jvm amongst all the jvms involved in my cluster. It >> > > looks like not too many classes make use of RT.nextID(), so I think my >> > > next move will be to add an e.g. RT.jvmID and use this in conjunction >> > > with RT.nextID(). If this resolves my problem then there is a good >> > > chance that my supposition was correct - otherwise it is back to the >> > > drawing board. >> >> > I think a better option is to make sure that the classloader you use for >> > deserializing is completely separated from the classloader Clojure uses to >> > load compiled functions. Then, no possibility of collision exists, >> > regardless of how you generate class names.- Hide quoted text - >> >> > - Show quoted text - >> >> That's an interesting idea, that I haven't really pursued, I think, >> OTOH, because client-side, I think that I want to be able to mix >> objects that herald from both local and remote jvms... - but this is >> just a gut feeling, I haven't really looked for any concrete usecases >> in any detail, so I'll give your suggestion consideration and see >> whether it would fit the bill. >> >> Using your approach would also be a good way of checking that my >> assumption about the name collision was correct, so I may see if I can >> set up my code to use a different ClassLoader in which to deserialise >> to see if that solves my problem. >> >> Thanks for this idea, it may help me a lot :-) >> >> Jules- Hide quoted text - >> >> - Show quoted text - > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
