ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hello, after three years of laying ground-work for a cross-platform database in form of many libraries (1), doing research about CRDTs and stretching core.async and other libraries as far as possible, I am happy to finally announce a first release of replikativ: replikativ is a replication system for confluent replicated data types (CRDTs). It is primarily designed to work as a decentralized database for web applications, but can be used to distribute any state durably between different peers with different runtimes (JVM, js atm.). Instead of programming thin web-clients around a central server/cloud, you operate on your local data like a native application both on client- and (if you wish to) server-side. You can also view it in reverse as a cloud being expanded to all end-points. You can write to CRDTs whenever you want and also access values whenever you want no matter if the remote peer(s) (servers) is available or not. In combination with our CDVCS datatype you can imagine it as git for data (expressed e.g. in edn) + automatic eventual consistent replication. https://github.com/replikativ/replikativ While there are still some issues and the design is not completely finished, I am pretty confident from our different design iterations and our running prototype that the current one can avoid race-conditions and is robust to errors. The interesting standard CRDTs are still missing, but I decided to first hear some feedback before growing the codebase and implementing optimizations. Let's build more open systems and share data, Christian (1) https://github.com/replikativ/ P.S.: The prototype https://topiq.es is currently hosted on a home server, if it loads too slowly, I will move it, but so far I felt a bit romantic about my basement and didn't want to spend money for an AWS instance or something else. Feel free to host your own instances and to connect them ;), it should be straightforward. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJWnpxRAAoJEKel+aujRZMk/uUIAMbAHVHOo0zNbbRr6QsapiLN ohQHTVqixkj/8qS3+z6ZmEGy572t2DH+QzXpHOqtqAS3mxGMikFKk078yWAYD3W3 QbZoxssDjgu/CGWsGAjuUetd8DoI1vI1T1oAVTo4IDo9uot5NEDHs3s5ZLB50NIX WOjm/muSPwkTt6B+oIp8ZsEYCH6RyLzTqkK6rOXxF0OoPv2XuK+TMgQJVzskmiaI 59Yf1TLizERN6DpyZbtFrWiVlgFF0+0K7GyW1qa7Bp7Yf9LE9yGra2WMjRwDdg7z 3SbZb5aXc/yaGztu+yN0wq3BWFCRSZQ9fh+VjrltGMA9BfExr53/aed1bIIcAtY= =v65c -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
Looks very interesting and I suspect there were some pretty hard problems to solve!!! Thank you for open sourcing this. Thomas -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
This is great work! Can it be compared to something like Datomic and Datascript? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 21.01.2016 06:22, Zubair Quraishi wrote: > This is great work! Can it be compared to something like Datomic > and Datascript? > It is mostly complementary with Datascript and it can be combined with Datomic as well. I have so far basically replicated the transactions with replikativ before applying them to a Datomic in memory database or Datascript so far. The main difference between Datomic and an eventual consistent system like this is that Datomic has one writer and therefore always a consistent view on the world, while CRDTs in general are eventually consistent. Even worse, to get a strong sequential order in CRDTs you have to introduce conflict management, so the git-like CRDT "CDVCS" that we have invented can replicate a transaction log, but needs conflict resolution in the case of multiple writers (different CAP theorem tradeoffs). We think there are many cases where this is acceptable. In section 5 are some thoughts so far: http://arxiv.org/pdf/1508.05545v1.pdf One idea is to use a CRDT like an OR-set for commutative transactions (e.g. mostly additions) and find a proper composition of CRDTs for the semantic of the distributed application you are building. The easiest way is to model something like Datomic where replikativ is distributing the transaction log to Datascript and you coordinate writes on a single peer (Datomic does a lot more like efficient index creation and distribution for you though...). That way you don't have to care about conflict resolution in the beginning and if your app grows you can find a proper CRDT composition. So in general you need to think a bit more about how you coordinate your writes and that you pick the proper datatypes for the semantics and write-load of your application, but you get writable replicas everywhere. Datomic alone is much easier to handle if you just need one central server and don't want to have offline writes for example. Om next can provide you some offline functionality, but this of course is limited by similar tradeoffs. A goal of this first release is to better understand how people approach state modelling in combination with Datascript and how replikativ can make this easier. We already have snapshot isolation to compose CRDTs, but I haven't implemented other CRDTs yet, as I would like to have your input first :). Did this comparison help you? Christian -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJWoVT0AAoJEKel+aujRZMkx9EIAJcL8YQbtbP79dguAszbzJdR iIkYydrrPIAFMF3XFaA65+1Yev97T4SiJbdpf4HBWhz7LUohHfbcz53h/xuVIKhj jf1JrIgJy7urN3w7WTuLoITQMHFszw6l5wlNm2QK9ijuDlougkfwFgcqu6uddIz2 /pyZ+Efzw/O6S9sLUtVwzKNeE5E1IdJUGuwO3GVyg/kTV3q8ag2mo992yodK7phb TkwlU5AYikPWCbBg2/daxbOM6Ussgk37n2Gg1VTAJI/B1o3mVpH/SVAbO8/2mKSW U3yfrX2LGNgaE9rteYF6kdB1tg+eTk8QsWO3BHSp4h8tbVbUPZHYpKc57NqSGLE= =APNm -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
Thank you, this looks immensely interesting! I'm still having trouble understanding when exactly one should use replikativ; are you saying it will make it easier to build offline applications? There's two things that make this difficult to understand: its academic nature, and the fact that many of us are just so used to the central server that it's hard to imagine anything else. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
Replicative lets you distribute state, so any time you'd like a distributed system to be able to share state, this is something you could consider to do that. In particular, every web application is a distributed system, so it's something you could use any time you wanted eventually consistent state between server(s) and clients. So the real question is what shared state offers. If you're not doing shared state, you're probably doing something where you send messages through some remote transactor that (ideally) ensures consistency of the system. But that means two things: * You can't be working offline (at least, not without a lot of work that starts to look like ad-hoc distributed state) * If you want optimistic UI updates (good for responsiveness & UX), you have to figure out how to undo UI updates if those transactions fail Replicative solves both of these problems. You can work offline on your copy of the data, and then when you're able to reconnect, it does a "merge" (analogous to a git merge). Ideally, the merge is clean and the remotes get copies of the work you did, and everyone is happy. But, if there are any kind of conflicting changes that need to be merged, you have to go through history conflict resolution process, again, quite like you would have to in git. So in very broad strokes, replicative is sort of like auto-synced git for application data. Chris On Thursday, January 21, 2016 at 9:00:28 PM UTC-8, Teemu Kaukoranta wrote: > > Thank you, this looks immensely interesting! I'm still having trouble > understanding when exactly one should use replikativ; are you saying it > will make it easier to build offline applications? > > There's two things that make this difficult to understand: its academic > nature, and the fact that many of us are just so used to the central server > that it's hard to imagine anything else. -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 22.01.2016 18:43, Christopher Small wrote: > Replicative lets you distribute state, so any time you'd like a > distributed system to be able to share state, this is something you > could consider to do that. In particular, every web application is > a distributed system, so it's something you could use any time you > wanted eventually consistent state between server(s) and clients. > So the real question is what shared state offers. > > If you're not doing shared state, you're probably doing something > where you send messages through some remote transactor that > (ideally) ensures consistency of the system. But that means two > things: > > * You can't be working offline (at least, not without a lot of work > that starts to look like ad-hoc distributed state) * If you want > optimistic UI updates (good for responsiveness & UX), you have to > figure out how to undo UI updates if those transactions fail > > Replicative solves both of these problems. You can work offline on > your copy of the data, and then when you're able to reconnect, it > does a "merge" (analogous to a git merge). Ideally, the merge is > clean and the remotes get copies of the work you did, and everyone > is happy. But, if there are any kind of conflicting changes that > need to be merged, you have to go through history conflict > resolution process, again, quite like you would have to in git. > > So in very broad strokes, replicative is sort of like auto-synced > git for application data. Yes, this description is pretty accurate :). There is no example for manual conflict resolution yet, but we already have a commit graph visualization to allow analysis and time travel (not implemented yet, but the vega.js plot routine should work in cljs apps): https://github.com/replikativ/flechtwerk Originally the plan was just to implement something like git for data. But over the last year I have refactored it a few times, so now any CRDT can fit and replikativ just covers the distribution for you. The C in CRDT most often stands for "conflict-free" (we prefer "confluent", because CDVCS represents conflicts in its value), so if you pick other CRDTs, e.g. sets or a LWWR you don't even have to do conflict resolution and just have an eventual consistent datatype. > > Chris > > > > On Thursday, January 21, 2016 at 9:00:28 PM UTC-8, Teemu Kaukoranta > wrote: >> >> Thank you, this looks immensely interesting! I'm still having >> trouble understanding when exactly one should use replikativ; are >> you saying it will make it easier to build offline applications? >> You might have a look at Nikita's very nice blog post: http://tonsky.me/blog/the-web-after-tomorrow/ He has a slightly more central take on distribution, as there still is one authority ordering the transaction log. Replikativ + CDVCS also allows you to fork data and dissent on its state, so you are allowed not to converge with another writer. But even if you only allow offline edits, you basically have to deal with conflicts, which is not really avoidable in his centralized design as well. >> >> There's two things that make this difficult to understand: its >> academic nature, and the fact that many of us are just so used to >> the central server that it's hard to imagine anything else. I am sorry for this appeal, I really tried to explain it well and give good examples including a non-trivial prototype application (https://topiq.es). Why do you think it is academic? I have written a paper and teamed up with the syncfree group for CRDT research, because many eventual consistent (NoSQL) systems promise things which they don't deliver and are poorly documented (read aphyrs Call me maybe blog series to get scared...). basho is in this research group as well btw. and is building state-based CRDTs into Riak. I wanted to put the ideas and concepts under serious scrutiny and give very high quality documentation about the core design. But my motivation is not academic, it is rather practical and political: I love data and I love machine learning, but the current trend is that data is always privatized. What can I improve to make it more clear that this a very practical project? Christian -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJWonqNAAoJEKel+aujRZMkZeYIAL5LC2wkPGh2prbq0ZHeIXOv WKFV1BA7uDF9IzfhGBK6Wgs0t++wsHQu+1ovZlDPM0XDliQ7/tIdONsZi2Cud15T oPrXVmyjJ8KEZXl9Skb9dEwVw79d8ZPYEdO75MoJB9+/bebad2H1ol1n3NEQYdVw GRbGLiGhOgJPzR5N/VVV8H/OFhtywxy0VYC9PRCB6N2EPpJvMPvBUyObhbGjEOCI ci2yNrsmA1lVRMoWtxoHlPKfOWdpRqM5Snr2gzGhB6xGa6lUYcF0t7xGy0zP6tcd luCtdtbcX1BTPu+xYmDJS9s2lP3L0xOiGblVTOv07Kjlgy6bwRNQL/LFRm/GOeU= =LY3W -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from t
Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
On Friday, 22 January 2016 20:53:25 UTC+2, Christian Weilbach wrote: > > >> There's two things that make this difficult to understand: its > >> academic nature, and the fact that many of us are just so used to > >> the central server that it's hard to imagine anything else. > > I am sorry for this appeal, I really tried to explain it well and give > good examples including a non-trivial prototype application > (https://topiq.es). Why do you think it is academic? I have written a > paper and teamed up with the syncfree group for CRDT research, because > many eventual consistent (NoSQL) systems promise things which they > don't deliver and are poorly documented (read aphyrs Call me maybe > blog series to get scared...). > I want to make clear that I didn't want to imply anything negative by "academic". Rather I meant that this topic is probably pretty foreign to many (or maybe it's just me), and this is the first time I've ever even heard of something called a "CRDT". In fact the project's github page immediately made me think that this is a real and practical tool that solves a difficult problem. :) I've actually read the blog post you linked many times, it's such a great, great post. I immediately thought of it when you announced replikativ, the blog post is a big reason in why I was so interested in this. > > basho is in this research group as well btw. and is building > state-based CRDTs into Riak. I wanted to put the ideas and concepts > under serious scrutiny and give very high quality documentation about > the core design. But my motivation is not academic, it is rather > practical and political: I love data and I love machine learning, but > the current trend is that data is always privatized. What can I > improve to make it more clear that this a very practical project? > Christian gave a great example a couple of posts up. To be honest, when I first read about replikativ I immediately thought about offline capability and rollback on failed transactions. The problem is that both of these can already achieved using (admittedly horrible) ad-hoc solutions, which can probably make people wonder whether they really need replikativ. It's unclear to me how much code I have to write to make the "git like" functionality you mention work (git is an awesome example by the way), and if it's a reasonable amount, why even abandon my previous ad-hoc solution? Again, this is not really what I think, just trying to put myself in the shoes of devs who don't understand replikativ yet. :) So to summarize: emphasize the git-like nature & offline and rollback capabilities. Show how much code it takes, and how many new concepts devs have to learn. This project is looking great, and it's clear to me that you're a very skilled maintainer who cares about the project! -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
http://lambda-the-ultimate.org/search/node/crdt nice to see powerful theory being made more practically available to us masses. ;-) -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
I just checked the About page at https://topiq.es/#. "You can run your own peer and extend the application through plugins or roll your own application on the same data!(base)." This kind of blew my mind. :) -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
Can you listen for changes on a server store? Also, have you developed a mechanism for just sharing parts of a distributed data structure? Chris On Tuesday, January 19, 2016 at 12:28:28 PM UTC-8, Christian Weilbach wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Hello, > > after three years of laying ground-work for a cross-platform database > in form of many libraries (1), doing research about CRDTs and > stretching core.async and other libraries as far as possible, I am > happy to finally announce a first release of replikativ: > > replikativ is a replication system for confluent replicated data types > (CRDTs). It is primarily designed to work as a decentralized database > for web applications, but can be used to distribute any state durably > between different peers with different runtimes (JVM, js atm.). > Instead of programming thin web-clients around a central server/cloud, > you operate on your local data like a native application both on > client- and (if you wish to) server-side. You can also view it in > reverse as a cloud being expanded to all end-points. You can write to > CRDTs whenever you want and also access values whenever you want no > matter if the remote peer(s) (servers) is available or not. In > combination with our CDVCS datatype you can imagine it as git for data > (expressed e.g. in edn) + automatic eventual consistent replication. > > https://github.com/replikativ/replikativ > > > While there are still some issues and the design is not completely > finished, I am pretty confident from our different design iterations > and our running prototype that the current one can avoid > race-conditions and is robust to errors. The interesting standard > CRDTs are still missing, but I decided to first hear some feedback > before growing the codebase and implementing optimizations. > > Let's build more open systems and share data, > Christian > > (1) https://github.com/replikativ/ > > P.S.: The prototype https://topiq.es is currently hosted on a home > server, if it loads too slowly, I will move it, but so far I felt a > bit romantic about my basement and didn't want to spend money for an > AWS instance or something else. Feel free to host your own instances > and to connect them ;), it should be straightforward. > -BEGIN PGP SIGNATURE- > Version: GnuPG v2.0.22 (GNU/Linux) > > iQEcBAEBAgAGBQJWnpxRAAoJEKel+aujRZMk/uUIAMbAHVHOo0zNbbRr6QsapiLN > ohQHTVqixkj/8qS3+z6ZmEGy572t2DH+QzXpHOqtqAS3mxGMikFKk078yWAYD3W3 > QbZoxssDjgu/CGWsGAjuUetd8DoI1vI1T1oAVTo4IDo9uot5NEDHs3s5ZLB50NIX > WOjm/muSPwkTt6B+oIp8ZsEYCH6RyLzTqkK6rOXxF0OoPv2XuK+TMgQJVzskmiaI > 59Yf1TLizERN6DpyZbtFrWiVlgFF0+0K7GyW1qa7Bp7Yf9LE9yGra2WMjRwDdg7z > 3SbZb5aXc/yaGztu+yN0wq3BWFCRSZQ9fh+VjrltGMA9BfExr53/aed1bIIcAtY= > =v65c > -END PGP SIGNATURE- > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 22.01.2016 20:35, Teemu Kaukoranta wrote: > On Friday, 22 January 2016 20:53:25 UTC+2, Christian Weilbach > wrote: > >> There's two things that make this difficult to understand: its academic nature, and the fact that many of us are just so used to the central server that it's hard to imagine anything else. >> >> I am sorry for this appeal, I really tried to explain it well and >> give good examples including a non-trivial prototype application >> (https://topiq.es). Why do you think it is academic? I have >> written a paper and teamed up with the syncfree group for CRDT >> research, because many eventual consistent (NoSQL) systems >> promise things which they don't deliver and are poorly documented >> (read aphyrs Call me maybe blog series to get scared...). >> > > I want to make clear that I didn't want to imply anything negative > by "academic". Rather I meant that this topic is probably pretty > foreign to many (or maybe it's just me), and this is the first time > I've ever even heard of something called a "CRDT". In fact the > project's github page immediately made me think that this is a real > and practical tool that solves a difficult problem. :) They are cool, but the basic concept is fairly simple. You just need a datastructure which converges if some operations commute. So think of sets or monotone updates (increasing timestamp). The fancyness comes when you combine these techniques to get better tradeoffs for certain cases, but they can't defeat the CAP limitations. So you cannot have a CRDT which models a strong sequential datastructure like a list and is conflict-free. What they really allow you to do is expose the real tradeoffs in form of datatypes instead of hiding it behind some "NoSQL" abstraction. > > I've actually read the blog post you linked many times, it's such a > great, great post. I immediately thought of it when you announced > replikativ, the blog post is a big reason in why I was so > interested in this. Yes, it is well summarized. :) Also Nikita is doing a great job with Datascript. > > >> >> basho is in this research group as well btw. and is building >> state-based CRDTs into Riak. I wanted to put the ideas and >> concepts under serious scrutiny and give very high quality >> documentation about the core design. But my motivation is not >> academic, it is rather practical and political: I love data and I >> love machine learning, but the current trend is that data is >> always privatized. What can I improve to make it more clear that >> this a very practical project? >> > > Christian gave a great example a couple of posts up. To be honest, > when I first read about replikativ I immediately thought about > offline capability and rollback on failed transactions. The problem > is that both of these can already achieved using (admittedly > horrible) ad-hoc solutions, which can probably make people wonder > whether they really need replikativ. It's unclear to me how much > code I have to write to make the "git like" functionality you > mention work (git is an awesome example by the way), and if it's a > reasonable amount, why even abandon my previous ad-hoc solution? I don't think you should abandon a working solution right now. replikativ is still very young and it is not ready for production yet. But for prototyping it can already significantly reduce code and allow you to focus mostly on the app (client-side) in my experience, letting replikativ take care of distribution and storage. > > Again, this is not really what I think, just trying to put myself > in the shoes of devs who don't understand replikativ yet. :) You are right to do so :). > > So to summarize: emphasize the git-like nature & offline and > rollback capabilities. Show how much code it takes, and how many > new concepts devs have to learn. Yes, I want to have interactive time travel in an app, this would be really cool. Hopefully I soon find the time to build it into topiq (or something else). Most developers would love this and it is one of the awesome things about Datomic. > > This project is looking great, and it's clear to me that you're a > very skilled maintainer who cares about the project! > Thank you :) Christian -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJWo6QfAAoJEKel+aujRZMk4eMH/iq9dovkdzkvfBciQHBKrgIA ru0ky39m59Ld/XpxXjDlb7FbV/Wcx+4zybTSuK/K2PnojOqx/ighRo+FMTOHGMhu SHQzXrkNq5jN7DUVzv9+pfv2LuGQGbFZFJt/kfHqG/igC4rNFC7F/F+JwAsOU/Wa 8T3yyg5oq51hwNAGxsOMHQNARh0oG3mP/vZ/U/3eCfeSVrdkg3F3JALlLAH+OEJs QjV7g/pv4WUuLJLsu+UWKV/lFJ4Ry5hbzSBaHDrYiFKbDwCPRfbwwyljdTwO4uNu YzT2bXhB+oRn4CzTq7e9sRdluQNn0IKoYWBeAuqfq54cl9OyI2ndp/zO0bigRss= =mxAA -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be
Re: [ClojureScript] Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
I've actually been looking at Replikativ (and CRDT's in general) over the past ~3 weeks, so I'm very happy to hear that Replikativ is moving forwards. It looks incredibly interesting. On Wed, 20 Jan 2016 at 16:30 Thomas wrote: > Looks very interesting and I suspect there were some pretty hard problems > to solve!!! > > Thank you for open sourcing this. > > Thomas > > -- > Note that posts from new members are moderated - please be patient with > your first post. > --- > You received this message because you are subscribed to the Google Groups > "ClojureScript" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to clojurescript+unsubscr...@googlegroups.com. > To post to this group, send email to clojurescr...@googlegroups.com. > Visit this group at https://groups.google.com/group/clojurescript. > -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ClojureScript] Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 20.01.2016 17:30, Thomas wrote: > Looks very interesting and I suspect there were some pretty hard > problems to solve!!! Since I started to explicitly use the CRDT formalism, I don't have to reimplement the wheel and there are a few pretty cool ideas out there, e.g. the CRDTs for text editing. Most important for replikativ is to make state transitions atomic (both in memory and on disk), so implementing CRDTs is fairly straightforward now (you can even do so in your application through a few protocols and a multimethod). In general getting there was a bit of work, since you can't just reuse JVM libraries or some databases. But this also reduced the dependencies, simplified the design and increased portability a lot. > > Thank you for open sourcing this. You are welcome :). Actually I think it has to be open-source, because what we really would like to do is share data between application and service providers and overcome the current Internet of data silos (both for political and for pragmatic business reasons). It was after some inspiration from EuroClojure 2013 that I recognized I really have to build this, but it was a long and often lonely journey, that is finally over :). I am curious about what the Clojure community can come up with in this area, as Clojure developers mostly open-source libraries, but not so many build open-source applications as for instance free-software communities in Python do. Concretely I hope to get some better picture of how to combine CRDTs with Datascript (or Datomic), so you can model nice views with React like Om next does and still have scalability and offline availability by picking the appropriate CRDT composition. In topiq for instance one can imagine to put single topiqs in OR-sets and remove the explicit conflict resolution for the sequential application of transactions of CDVCS since operations commute, while using a CDVCS with strong consistency for a personal social network profile etc. Christian -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJWoU7AAAoJEKel+aujRZMkYBsH/1pM8qEnnAhbXAmSwunvc6HR chIzihaM6I4IdXWjB7KFj78lejwS8/ang/038qUNxLu+5R/4e26f0n2hZg9Xdywu FEWY7dq5s+D7bODGYtNuW9aNDZwscfls/tFoMs1DZ80RrYQzMYBlsHWE+V6LJ6Co d1+jC8l0MZKQJ/xiXHB3/21xafPRbqoDj+lUV61Tz4ojgjX33vSysm5JvqgVM/Uy Sbji2Wmbd45tpRjp7B8x5nzP5akshJv3tH9VtulTEyOh2CdasPF7PxPg2CG67AFH Zapw92RB9FZ2U0nbV5/S6vD3p6nKgKPGeKqEi9c9uwd0mm3+iCfSfxl8ky7BB3I= =ukOj -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ClojureScript] Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
> > I am curious about what the Clojure community can come up with in this > area, as Clojure developers mostly open-source libraries, but not so > many build open-source applications as for instance free-software > communities in Python do. > > Just reading this and Braid came to mind: https://github.com/braidchat/braid Maybe that is something we can show off some Clojure and ClojureScript. Haven't had time yet to look at this myself yet. Thomas -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ClojureScript] Re: ANN replikativ 0.1.0 - strong eventual consistent P2P replication for clj and cljs
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 23.01.2016 05:11, Christopher Small wrote: > Can you listen for changes on a server store? What exactly do you want to do? If you want to track updates to a CRDT, I would create a stage and subscribe to the updates as I do in the topiq client: https://github.com/whilo/topiq/blob/master/src/topiq/core.cljs#L177 This is still very "low-level" and I want to provide a proper function for this, but maybe you get the idea. I can refactor it and put it into replikativ, if this helps you. > > Also, have you developed a mechanism for just sharing parts of a > distributed data structure? Do you mean metadata? All metadata needs to be distributed for the code to work (the commit-graph to be connected etc.), but not the data (commits and transactions). The syncfree group did some work on partial metadata replication with swiftcloud: http://arxiv.org/pdf/1310.3107.pdf They need a fixed number of replicas in the datacenters for their vector clocks, this is not possible in an open p2p system. But I am open to optional metadata reduction strategies with the help of some centralized coordination, e.g. physical time, vector clocks or a classical coordination service. Atm. even all data needs to be backed up by each peer, but I came to the conclusion that this is not necessary, peers can replicate the metadata only and updates from them will then be dropped by peers which need the data, but can't get it from them. I just wanted to start with the assumption that storage is infinite and I don't need to worry much about it (but bandwidth is not ofc.). Christian -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJWo6EBAAoJEKel+aujRZMkurwIALavrxwYx7/wfkEp66NUS1Mu G3XFUxgpT/yQZ4zT/nO9+xlB+6X4ZFGJs+4beLVR8vc6rY+2WrBUwU6O8kxA8BIV Tt3ZZTUlSh8gxXUiYqO9XGmU605nbpdRYv2FA5xi1kTrmd5tZ2SsjaSMPB6SHbYE MMqk6LZyxvrrYHizFbopVKdLP3awkCBCUbjVNZm7w8mLtxnTafItiumCepFtqZw6 AOGDbNWkdDG5f39cTTNShfnLE04XFl+b5AwqvSyXB8OfB6bi/w4toYrxXbGV3T/n e/9tEvrfdsrc6zoHi4sQf8EdGEU5p75acrvb9Z1bbcMeiBhe5G87DwN3SP9KXfo= =lb1U -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
