Thanks for the note about operational transform (OT). I had not heard of it before. From some casual reading, OT appears to focus on the problem of collaborative document editing. I assume that Share.js extends this to structured json? This is definitely something I think a lot of people would be interested in. I'm not entirely sure how one would apply it to a case where a web app is backed by an sql database though, or if it is even appropriate for all use cases. The wiki page for OT mentioned differential synchronization as a (possibly easier to implement?) alternative. It appears to be based on diff/match/patch, which is probably also extendible to clojure data structures.
http://en.wikipedia.org/wiki/Operational_transformation https://neil.fraser.name/writing/sync/ as an alte Thanks for the links, I'll have to read more on these topics. Regarding our sql view in the browser idea, it turns out that we can in fact get away without a transactor. You can simply put a lamport clock in the database itself. So long as the db transaction engine serializes the transactions, this clock can then be used to detect out of order updates on the client. On Sunday, February 23, 2014 12:19:47 AM UTC-5, Joel wrote: > > > I'm interested in this thread, it does seem however that DerbyJS is > farther ahead in this type of work though. I was looking for a similar JVM > stack, but not sure how to put it together as well. Atmosphere seems like > it fits for the basic communication even including pubsub. Derby updates > the view through OT (operational transforms), with types for both JSON and > for Text. The code could be rewritten to work in Clojure using core.async, > etc. I'm not sure how it does views/authorization though, or how does OT > work with undo? > > DerbyJS also seems to have a relatively pluggable system, compared to say, > Meteor, but pluggable takes more work and tends to make things more generic. > > J > > > On Thursday, February 20, 2014 3:19:55 PM UTC-6, Alexander Hudek wrote: >> >> (Dave and I work together). >> >> I suspect it will be easy to come to a reasonable license. My own >> preferences would be EPL or MIT, and EPL only because it's common in the >> community. >> >> Beyond that, something like chord isn't actually the big problem in a >> scheme like this. The real issue is ensuring that data replication is >> reliable. Here are the main pain points we've identified so far: >> >> 1. Updating a view on the client should update the source data. >> >> This is called the "view update problem" and is fantastically difficult >> to solve. There are lots of publications on this (e.g. >> http://www.ijcte.org/papers/278-D038.pdf). I would recommend that we >> punt on the problem for the near term and settle for explicit >> update/insert/delete actions. >> >> 2. SQL, keystore (nosql), datalog (datomic). >> We are approaching this from the SQL perspective. You would define and >> name a view on the server along with it's sql. Clients would then be able >> to register to have a view be mapped to an atom and kept up-to-date. >> Presumably a keystore can be treated conceptually as a single table. I >> haven't thought about what a view would mean for datomic, nor how to keep >> one up-to-date. >> >> 3. Keeping the views up-to-date. >> This is the hardest part. Here we want all the client views to be >> automatically updated, and to be consistent with the source data. There can >> be some amount of time lag between when the source data is updated and when >> the view is updated. We've worked out some ways to do this for a subset of >> SQL queries (including joins). It does require that queries are given in >> something like honeysql though. To compute update deltas for views you need >> to do some manipulation of the original query. >> >> The other part of this is making sure that view delta updates arrive in >> the right order. This becomes especially important when you have a >> multi-server setup (e.g. high availability or load balancing). It's the >> same problem that all the distributed databases need to solve. The only >> reasonable solution we've thought of is to linearize the updates via a >> single write transactor mechanism. This would likely also need to be >> pluggable: some people will want something simple such as a thread in a >> single server, others will need something heavier such as a service sitting >> on an internal node with a traditional tcp/ip interface. >> >> This doesn't seem ideal, but I don't know of any better way at the >> moment. I'm not sure that things like vector clocks would work for all data >> sources. >> >> 3.5 Pluggable communications layer. This is where chord fits. But it >> could also be handled by amqp, browserchannel, or other things. It's >> reasonable to use core.async as part of the pluggable interface. >> >> 4. Error handling, recovery, security. The security aspect definitely >> needs to be a pluggable component. >> >> >> On Thursday, February 20, 2014 1:52:19 AM UTC-5, Mikera wrote: >>> >>> >>> >>> On Thursday, 20 February 2014 13:56:29 UTC+8, David Della Costa wrote: >>>> >>>> > It's not clear to me that the server side should be tied to Om >>>> > specifically. It seems like the requirement is more to have a server >>>> > component that can pass messages / state changes in a generic way to >>>> > clients (probably using core.async channels, with the ability to use >>>> > websockets etc. as the underlying transport). This server component >>>> > would be useful for all kinds of clients (not just Om, not just >>>> > Clojurescript, maybe even other server systems). >>>> > >>>> > I've hacked some stuff together that does this kind of thing for >>>> > specific projects, but it would be nice to get a solid standard >>>> > library in the ecosystem for this. >>>> >>>> We have a concrete need for a system like this and in fact already have >>>> a prototype in production (which uses browserchannel as opposed to >>>> websockets, but these should be interchangeable) with ampq messaging >>>> and >>>> http wrapped in core.async and Om on the front-end, but we are working >>>> through some issues relating to scaling while providing eventual >>>> consistency across a number of web front-ends. The end goal is >>>> essentially materialized views in the client. >>>> >>>> We would also be very interested in working with other folks to provide >>>> something with general applicability to the whole community. Please >>>> let >>>> us know if you'd want to contribute! >>>> >>> >>> I'd be happy to contribute to a library that met the following >>> requirements: >>> - A sensible open source license (LGPL or EPL are fine) >>> - As simple as possible (minimal dependencies and API surface area) >>> - Designed for plug+play with the related libraries in the ecosystem >>> (core.async, http-kit, ring etc.) >>> - Reasonable community consensus that it is a good design >>> >>> BTW chord looks pretty promising as a foundation / model for some of >>> this kind of work >>> https://github.com/james-henderson/chord >>> >> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
