Another good question, and the answer is "it depends". As a guiding principle for the expectations of this abstraction, convergence should yield a collection as if it had all of the operations applied to it in the order that they were applied, locally. So let's work with M, and nodes A and B.
A creates M as a simple map {:foo true} and replicates it to B. A has M ({:foo true}), and B has M' ({:foo true}). A performs (assoc M :bar false}. A has M ({:foo true, :bar false}) and B has M' ({:foo true}). B performs (dissoc M' :foo). A has M ({:foo true, :bar false}) and B has M' ({}). B sends M' to A, and A synchronizes it. A has M ({:bar false}). B has M' ({}). A sends M to B, and B synchronizes it. A has M ({:bar false}). B has M' ({:bar false}). That's a simple case because A and B are modifying different keys and converge to the same value without contention over those keys. But what if A had done (assoc M :foo false) instead? There's still a synchronization conflict between the two, but the key :foo has been put in contention between A and B. In schism, this is resolved by the last writer winning, because that is what would happen on an isolated node working with a data structure locally. On Thursday, April 19, 2018 at 1:46:41 PM UTC-4, John Newman wrote: > > Alex, yeah that explains it for me. I'll probably want to use a fully > connected address space, via a mesh or a hub and spoke topology, that I > fully manage - so this is the perfect level of abstraction for me. I'd like > to see that ZQ implementation though. > > So, to be clear, if node A sends schism-map M to node B, both A and B > update M, B sends M back to A, A converges M1 and M2: If M1 and M2 have > destructive conflicts, does the most "recent" change win? Or does the local > copy win? > > Thanks, > > V/r > > John > > John > > On Thu, Apr 19, 2018 at 8:55 AM, Alex Redington <aredi...@gmail.com > <javascript:>> wrote: > >> I'll try to answer this question and John's at the same time. >> >> Schism does not try to manage state for you over time; there are a lot of >> good tools for doing that already (atoms, refs, agents, channels, etc). >> Schism is a set of collections that augment basic clojure collections with >> enough additional information that you can take two persistent collections >> with common ancestry and merge them together. So, you could take a schism >> map held in an atom, dereference it and send it from a pedestal process to >> an om application as edn, and read-string it in the om app to have a >> replicated copy of the map. The om app could make some changes (dissoc one >> key, update another) without communicating to the pedestal app for each >> operation. Then send it back to the pedestal app in a POST body, and the >> pedestal app would swap! the atom using schism.core/converge and the value >> it just received from om. The changes, both additive and destructive, would >> be replicated in the atom's value. >> >> So, yes, you could build a serverless chatroom where each client held an >> atom with a schism collection, and they sent their copies around using >> WebRTC to stay in sync. You could push a schism edn serialization over a >> message queue to be synchronized on the other end. I anticipate pushing >> each discrete state over the mq with an add-watch hook might yield bad >> performance for little semantic gain. Schism collections are necessarily >> more expensive than Clojure collections in serialization size, and >> synchronization is where all of the signficant computational expense of >> these kinds of data structures resides, so doing it less frequently than >> every discrete update will probably be best. (A debounced send & sync that >> guaranteed transmission after n milliseconds of being inert would make >> sense to me) >> >> I'm not presently confronting a problem that these solve in my day job; >> my intent was to build and have the tool ready if and when we wanted to >> have better answers for "offline sync" in a single page ClojureScript >> application. >> >> Thank you for the great questions! >> >> -Alex >> >> On Wednesday, April 18, 2018 at 9:55:36 PM UTC-4, Luke Burton wrote: >>> >>> >>> This is cool! >>> >>> There seems to be some room for a recommendation on how to apply these >>> data structures effectively in practice, especially around the distribution >>> mechanism. For instance, I can imagine storing these in an atom and using >>> watchers to push changes out over zero-mq. Or is that the Worst Idea Everâ„¢? >>> I feel like the transit layer probably has *some* impact on whether CRDTs >>> are the right solution to the problem, and I'm interested to know what >>> those boundaries are. >>> >>> Have you used them in production over the network, and if so, what >>> solution did you use? >>> >>> Thanks again! >>> >>> >>> >>> On Apr 18, 2018, at 5:57 PM, Alex Redington <aredi...@gmail.com> wrote: >>> >>> Good evening! >>> >>> I submit for your evaluation and reasoned feedback a library I've been >>> working on to provide a set of convergent replicated data types to Clojure >>> and ClojureScript: >>> >>> https://github.com/aredington/schism >>> [com.holychao/schism "0.1.0"] >>> >>> Schism provides convergent collections for sets, vectors, lists, and >>> maps, along with edn readers and writers out of the gate. My intent is to >>> provide you with all the tools necessary to replicate a collection across >>> two or more computing nodes. >>> >>> Replication is a hard problem, so there are a few caveats to these tools >>> as I provide them: >>> >>> - You must identify nodes within your cluster. You may choose to >>> identify nodes with a random UUID, in which case schism will help you to do >>> so. Node identifiers must be clojure serializable data. >>> - Schism purposefully avoids carrying around a monotonically increasing >>> historical collection of data about deleted entries. Consequently there are >>> some ambiguities during convergence that may not exactly mirror local >>> modification. >>> - Schism is only solving the problem of synchronizing two in memory >>> sets. Maintaining identity of those two sets, tracking state changes, and >>> long term durability are responsibilities left to the schism user. >>> >>> Schism collections are persistent collections, so you should feel free >>> to work with them as a drop in replacement for a function which would work >>> against a Clojure collection. The usual utilities such as conj, assoc, >>> dissoc, disj, and rest are pure functions which will return you derived >>> copies, implicitly soing the convergence bookkeeping necessary in the >>> background. As you work with it, schism will maintain node and timestamp >>> information, with the goal of convergence providing the same result as if >>> all previous invocations had occurred on one local collection in memory. >>> Schism's requirements are your expectations of a Clojure collection, so >>> hash, =, and support for meta are all included, as well as many other >>> functions defined against Clojure's own collections. Particularly with hash >>> and =, you can expect a schism collection to return the same hashcode as >>> its Clojure equivalent, and to determine equality in the same way as its >>> Clojure equivalent would. >>> >>> I've built schism in the absence of a direct itch to scratch with it. >>> Should you find that it betrays your expectations of Clojure collections, >>> I'd greatly appreciate a bug report and will work to correct schism quickly. >>> >>> I hope that it assists you in solving your own problems. >>> >>> -Alex >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To post to this group, send email to clo...@googlegroups.com >>> Note that posts from new members are moderated - please be patient with >>> your first post. >>> To unsubscribe from this group, send email to >>> clojure+u...@googlegroups.com >>> For more options, visit this group at >>> http://groups.google.com/group/clojure?hl=en >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "Clojure" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to clojure+u...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clo...@googlegroups.com >> <javascript:> >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+u...@googlegroups.com <javascript:> >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+u...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.