Hans Hubner's BKNR framework for CL explores this in a very interesting way - while relying on CLOS meta-object protocol the ideas could prob. be extended to Clojure. With some ABCL interaction this would make CL -> Clojure || Clojure -> CL interop possible at the JVM level with persistence...
Someone call me when i can pull my head out of the pipe :) s_P On Dec 18, 8:10 pm, r <nbs.pub...@gmail.com> wrote: > On Thu, Dec 18, 2008 at 11:06 PM, Chouser <chou...@gmail.com> wrote: > > > I've pondered a couple approaches, though only enough to find > > problems. > > > One approach would work act like a Clojure collection, with structural > > sharing on-disk. This would be great because it would have > > multi-versioning and transaction features built right in. It would > > also have the potential to cache some data in memory while managing > > reads and writes to disk. > > > But Clojure's persistent collections rely on garbage collection -- > > when old versions of the collection are no longer referenced, the JVM > > cleans them up automatically. How would this work on disk? How would > > you define "no longer referenced"? > > [1. just to confirm I understand you correctly: is this a > "transparent" case where each newly constructed data structure is > immediately written to disk?] > > I can see another problem here: assuming we have multiple "active" > versions of such structure at the time of closing the application, > which of them should we restore when the application restarts? Do we > need explicit "save"&"restore" commands to mark the data we want > anyway? Or is it better to automatically keep track of "heads" per > (named) thread? Or use a (transparent) Ref? > > Another issue is the efficiency of the GC itself. Whatever scheme we > use, Clojure is going to use GC a lot. This may cause strong > fragmentation of data on disk (not to say about performance penalty). > > > Another approach would be at the Ref or Agent level, where watchers > > could be hooked in. (Watchers are currently only for agents, but are > > planned for refs as well.) Watchers are functions that are called > > when their underlying mutable object has a change committed, so they'd > > be able to sync the disk up with new in-memory value. But this means > > the whole collection would have to be in-memory. Also the watcher > > gets no hint as to *what* in the collection changed. > > > So for now it seems we'll have to make do with "normal" mechansims > > like SQL libraries. > > [2. is this an "opaque" case where data sit in memory and only when we > switch the Ref the on-disk representation is updated?] > > Hiding the whole data structure behind the Ref and syncing data only > when it changes would solve both problems mentioned above. The cost is > that the data would not longer be updated incrementally so the whole > structure would have to be flushed to the disk. This wouldn't be very > efficient but would work with existing database back-ends. > > Perhaps it would be better to use a combination of 1. and 2., i.e. not > only to hide data structures behind a Ref and commit changes to disk > only when the Ref changes (as in 2.) but also to use a journal for > tracking "modifications" to data so that only incremental changes have > be done. Such journal could be translated (and optimized) into a bunch > of INSERT&DELETE commands. > > Another question: what kind of data structures? List, vector and hash > or maybe a new specialized type (a table)? Performance characteristics > are going to be very different from in-memory data so maybe it makes > sense for the whole mechanism to be opaque. > > -r. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
