I'll go one step further and say that we shouldn't have to call "persist namespace". It should be automatic such that a change to the state of an identity is transactionally written.
Let's start with refs. We can tackle the other identities later. The API is simple. Call (refp) instead of (ref) when creating a persisted ref. Passed into the call are a persistence address (file path, DB connection string, etc.) and a name that has to be unique to that persistence address. Not all refs end up being referred to by a top-level symbol in a package, and multi-process systems are hard... Ensuring uniqueness of name is up to the programmer. Upon creation, Clojure checks to see if the refp exists in the store; if so it instantiates in memory with that state, else it uses the default in the call. In a dosync block, the function runs as normal until commit time. Then Clojure acquires a transactional write lock on each refp that is alter-ed or ensure-d. It checks the value against memory. If it's the same, commit data store changes. If not, retry after refreshing memory with the current contents of the store. If the data store commit fails, retry a number of times. If the data store commit still can't proceed, roll back the whole thing. commute and refset are slightly different, but for an initial implementation, just treat commute as an alter, and ignore refset. Does this make sense? My intention is to cover the 80% case. The implementation would necessarily be chatty, since the API is chatty. That's OK for most systems. This API has the benefit of being able to be shared across Clojure instances. It's a nice bonus. A dosync block may contain symbols pointing to refp's spanning different data stores, which isn't too hard to handle. It simply requires that if this is the case, each data store must support two- phase commit or some other distributed transaction supporting protocol. For an initial implementation, I would just throw an exception. I've begun working on an implementation using BDB. What do people think? On Aug 30, 5:02 pm, nchubrich <nchubr...@gmail.com> wrote: > I'm not aware of any, but +1 for seeing persistence handled as part of > the language. A big project and a long-term one, to be sure, but > could it not be considered a goal? > > In my student days, I was talking to a well-known Lisper (name > suppressed for fear of Google indexing) about some data structures in > MIT Scheme. When I asked about saving them to disk, he said in > effect, "You're on your own----that's something that \should be > handled, but never is". > > I think people are so used to this state of affairs they forget how > ugly it really is. Programming languages are like Moses without > Joshua: they lead your data in the wilderness, but when it comes to > finding it a permanent home, you have to talk to someone else. And > these "someone elses" (who seem to be as numberless as the sons of > Abraham) each have their own habits and ways of talking. > > Persistence libraries always end up warping the entire codebase; I've > never succeeded in keeping them at bay. Using data with Incanter is > different from ClojureQL, which is different from just using > contrib.sql, and all of it is different from dealing with just > Clojure. (I've never even tried Clojure + Hibernate.) You might as > well rewrite the program from scratch depending on what you use. > Maybe other people have had better luck; but whatever luck they have, > I'm sure it is a fight to keep programs abstracted from persistence. > > I'd like to be able to work with mere Clojure until my program is > complete, and then work in a completely separate manner on how to read > and write data. Or maybe there would be off-the-shelf solutions I > could plug in for different needs: low latency, high read, high write, > large stores, etc. > > On the Clojure side, you would simply call something like "persist > namespace", which would save the state of your current or given > namespace (unless you pass it the names of variables as well, in which > case it only saves those). And to read data, you would simply require > or use it into your namespace: you could choose what granularity to > give first-class status: just tables, or columns as well, etc. And > you could do this equally well for XML, JSON, relational data, or a > graph store; your choice. And the only difference between these and > ordinary variables would be----heaven forbid!----a disk access might > happen when you deal with them! > > To have such a system work well, you would need to enrich the way you > query Clojure datastructures. I have some ideas on that, but I'd like > to make sure I'm not shouting in the dark first. > > I'd like to see a day when programmers need to worry about persistence > about as much as they worry about garbage collection now. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
