You're right Rich, We all have to agree on the means used to implement this in the Clojure runtime. Any code we throw right now has to be somewhat aligned with these decisions.
The decision to hide the AtomicReference class was easy to take. It was an unavoidable obstacle. Any other issues from the Clojure run time will need more thinking and there might be other ways to do work around these issues. I can post an update each 2 weeks or so or on demand before we spit out code if we face an issue. Right now we are busy writing a tool in Clojure to generate the terracotta configuration from a set of classes. Finding each dependent class one by one is too slow. We always end up missing one. It will also spit out the locking section with all these member functions. This tool will speed us a lot in getting the prototype up. We will get the master pieces of the XML configuration. We can them removing unwanted classes from the configuration and tune the locking strategy as you test things. Luc On Fri, 2009-02-27 at 18:05 -0800, Rich Hickey wrote: > > > On Feb 27, 6:54 pm, Luc Prefontaine <lprefonta...@softaddicts.ca> > wrote: > > Having the ability to redefine a function once for all instances is > > something we really want... > > and you need name spaces to be shared for that to happen. > > We take the approach of sharing everything that seems to worth it, then > > we will see > > what we might need to keep private to each JVM. > > > > Sharing var roots and refs also is something we want so we can initiate > > STM transactions > > involving multiple JVMs. > > > > We have much to learn from a prototype and that is what we strive to > > achieve the immediate future. > > After this step, real fun will begin, right now were having only an > > appetizer... > > > > Luc > > > > It would be great if you could mention the difficulties you face as > you go, before you spend too much time on workarounds. I am interested > in seeing Clojure on Terracotta and if there are things I can do > easily to support this effort I will. > > Rich > > > > > > > On Fri, 2009-02-27 at 16:51 -0500, Paul Stadig wrote: > > > Yeah, after sharing clojure.lang.Keyword.table I tried to share > > > clojure.lang.Namespace.namespaces, but ran into a problem because > > > Namespace uses an AtomicReference for its mappings and aliases > > > members. I thought that not sharing namespaces would be a problem (and > > > maybe it still is I don't have as much practical experience with this > > > as you), but I wasn't sure it would be a problem. Looking up a var in > > > a namespace on different JVMs would find different vars, but since > > > they are thread local, then I'm not sure its an issue. Having all of > > > your refs shared without having to specify each specific ref would be > > > interesting, but since most of the stuff (functions, vars, etc.) are > > > immutable or thread local, then I'm not sure how much of an issue it > > > is. > > > > > Obviously, if you were going to redefine some functions or something, > > > then you'd either have to do so on each JVM, or just restart all of > > > the clients. > > > > > And as I said in my article, I didn't do any work with agents, so > > > maybe there's a lot missing from my part of the puzzle. > > > > > Paul > > > > > On Fri, Feb 27, 2009 at 4:37 PM, Luc Prefontaine > > > <lprefonta...@softaddicts.ca> wrote: > > > > We are trying to get Clojure shared over Terracotta, not just specific > > > > things but the whole Clojure object space > > > > (name spaces, root values, ....) except stuff that needs to remain local > > > > (streams, ....). > > > > > > We take an all or nothing approach here, we would to see many Clojure > > > > instances work as a single > > > > entity. > > > > > > We are facing a number of issues, one of them being the lack of support > > > > by > > > > Terracotta of AtomicReference. > > > > > > Terracotta has placed this class in the list of classes they will > > > > eventually > > > > support but has of 2.7.3 > > > > it's still not supported. > > > > > > The Sun AtomicReference does peeks and pokes in the heap using internal > > > > routines of the JVM (JNI). > > > > Clearly this cannot be done with multiple JVMs, different object heaps, > > > > .... > > > > > > Terracotta suggest using the oswego library but since Java 5 has been > > > > out, > > > > it's in maintenance mode only and > > > > integrating another library to the puzzle for that single purpose did > > > > not > > > > look to us as very efficient. > > > > > > So we created a SharedAtomicReference implementation that uses standard > > > > locks to control access to > > > > the value with the corresponding Terracotta configuration. > > > > > > We use a factory to decide at run time which implementation to use > > > > based on > > > > a system property. > > > > To preserve the Clojure code base, we implemented an AtomicReference > > > > interface and a delegate to > > > > the AtomicReference class. The Clojure code uses now interface. > > > > This allows us to make progress without disturbing the Clojure code > > > > base too > > > > much. > > > > > > This has some performance implications that we do not fully understand > > > > yet > > > > since we need a living Clojure implementation > > > > to work with, something we are in the process of creating... what a > > > > catch 22 > > > > :))) > > > > We thought about spending time working on this aspect right now but we > > > > prefer to wait for the first alpha release. > > > > > > As for the rest of the Clojure code, we are working on the classes > > > > implementing vectors, hash maps, .... > > > > to get them shared through terracotta. > > > > > > Our main challenge these days is putting together the terracotta > > > > configuration of the internal classes of Clojure that need to be shared. > > > > > > We may hit issues that make some classes non portable. > > > > These will require us to implement an alternative. We will use then a > > > > slightly different approach, > > > > we need only factories to allocate an alternate version or the "local" > > > > implementation. > > > > The Clojure code already uses interfaces to access the data objects so > > > > this > > > > will be almost transparent in the code. > > > > > > We prefer to re implement an "distributed" implementation side by side > > > > with > > > > the original one and easily switch between them > > > > to compare behaviour and performance with the same code base. > > > > > > When we have a clearer understanding of how it performs we will look at > > > > how > > > > to merge this in the code base > > > > that will have changed in between. We may be able then to reduce the > > > > number > > > > of alternate implementations > > > > if more where created. > > > > > > The work is in progress, next week is a school break week here so the > > > > pace > > > > will slow down a bit. > > > > We wanted to start earlier on this (before XMas) but we had updates to > > > > put > > > > in production , client caring to do, ... > > > > and other mundane tasks to get the bread and butter on the table. > > > > > > Comments/suggestions are welcomed... > > > > > > Luc > > > > > > On Fri, 2009-02-27 at 12:20 -0800, Paul Stadig wrote: > > > > > > I've recently done some experimentation with Clojure and Terracotta. > > > > I've detailed my experience at: > > > > > >http://paul.stadig.name/2009/02/clojure-terracotta-yeah-baby.html > > > > > > and shared my code at: > > > > > >http://github.com/pjstadig/terraclojure/tree/master/ > > > > > > I'm the first to admit that I'm not an expert in Terracotta. This is > > > > actually my first time working with it. > > > > > > I was able to setup a permanent store, and to run a couple of > > > > transactions against some refs shared across multiple JVMs. I ran into > > > > two snags. The first was that Keyword did not implement hashCode, > > > > which was a simple fix. Rich graciously added a hashCode method. :) > > > > > > The other was how to point Terracotta to a var from the Java side. I > > > > ended up using a simple Java class to hold on to my refs, since then > > > > it was easy to point Terracotta to the right place. It works just > > > > fine, but I'm not sure if there is a better way to do it. > > > > > > I'd be interested in knowing if anyone else has experience to share > > > > about using Terracotta with Clojure. > > > > > > Paul Stadig > > > > > > -- > > > > > > Luc Préfontaine > > > > > > Off.:(514) 993-0320 > > > > Fax.:(514) 993-0325 > > > > > > Armageddon was yesterday, today we have a real problem... > > > > -- > > > > Luc Préfontaine > > > > Off.:(514) 993-0320 > > Fax.:(514) 993-0325 > > > > Armageddon was yesterday, today we have a real problem... > > > -- Luc Préfontaine Off.:(514) 993-0320 Fax.:(514) 993-0325 Armageddon was yesterday, today we have a real problem... --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
