It often happens that I would like to have gaurantees about the
consistency of in-memory data structures and external resources.  For
instance, in the last two large systems I've built, the system would
respond to an external message received via socket, do some complex
processing based on the contents of the message which modified it's
internal state, write a log of the message to a database and update
some other database tables so that the internal state of the system
was reconstructable in case of crash, and then send some other
messages to clients via socket.  This all had to either work, or not
work, with full ACID properties for the system as a whole (both the
database and memory).  This was an enormous pain, requiring in-memory
rollback functionality to be coded by hand for each and every possible
interaction.

It seems like it should be possible to extend the STM semantics of
Clojure to allow this sort of thing to Just Work.  From the user point
of view, one would call dosync-external, rather than dosync, and pass
it an external (JTA?) transaction manager along with the operations to
be executed.  If either the in-memory operations or the external
operations required rollback, then both would be, otherwise both would
be committed.  If committed, any agent sends that the transaction had
sent would occur.  Simple to use, and pretty much exactly what's
required for a high-reliability message processing engine.  Literally
thousands of lines of code would disappear from each of my last two
projects.

Looking over the current implementation of Clojure, I think I've
convinced myself of the following facts.

1) This can't be done purely in Clojure code itself, as the necessary
hooks into the STM don't exist (nor, I would argue, should they).

2) This could be done pretty easily by hacking the Clojure runtime, at
a (obviously excessive) cost of inducing a hard dependency between the
core runtime and some particular transaction manager.  The STM
implementation already has enough of a two-phase commit structure to
make it pretty easy to integrate with any given transaction management
system.

3) Creating some sort of plugin system so that arbitrary transaction
managers could be integrated with the Clojure STM runtime is doable,
but doesn't much fit with the current architecture.

4) If one were to do such a thing, it would be pretty easy to extend
the STM with features like timeouts and such.

Thoughts? Am I missing anything?  Has this already been attempted in
some way I was unable to google for?  My temptation is to try to
implement something as a quick spike and see how it works.

--Dave Griffith



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To post to this group, send email to clojure@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/clojure?hl=en
-~----------~----~----~----~------~----~------~--~---

Reply via email to