On Wed, Mar 22, 2017 at 3:33 PM, Mirage Abeysekara <[email protected]> wrote:
> Hello Edward, > > Why do we need a distributed locking mechanism, if the gossip provides > eventual consistency model? > Can you please give me a bit of explanation or simple use-case? > > Thanks > > > *Mirage Abeysekara* > Undergraduate > Computer Science and Engineering > University of Moratuwa > Twitter: https://twitter.com/MiRAGECreator > GooglePlus: https://plus.google.com/u/0/+MirageAbeysekara > > On 18 March 2017 at 21:50, Edward Capriolo <[email protected]> wrote: > > > On Thu, Mar 16, 2017 at 5:16 AM, Mirage Abeysekara < > > [email protected]> > > wrote: > > > > > Hi Edward, > > > > > > I updated the jira ticket with design documentation link. > > > The time period for GSoC is three months. I would like to be work with > > > another ticket. Can you assign me for a suitable ticket? > > > > > > Thanks > > > > > > *Mirage Abeysekara* > > > Undergraduate > > > Computer Science and Engineering > > > University of Moratuwa > > > Twitter: https://twitter.com/MiRAGECreator > > > GooglePlus: https://plus.google.com/u/0/+MirageAbeysekara > > > > > > On 13 March 2017 at 18:47, Edward Capriolo <[email protected]> > > wrote: > > > > > > > I assigned the ticket to you. I labeled it GSOC 2017. Please > link/copy > > > your > > > > design docs into the ticket. It would be good to have another ticket > > > > earmarked to you. I do not know how long GSOC is but with the speed > you > > > did > > > > the last ticket I do not anticipate pub/sub taking a month. > > > > > > > > On Mon, Mar 13, 2017 at 6:16 AM, Mirage Abeysekara < > > > > [email protected]> > > > > wrote: > > > > > > > > > Hello Edward, > > > > > > > > > > I updated the document with changes. Here is the expected design of > > the > > > > > classes. > > > > > https://drive.google.com/file/d/0B8l-mZ- > > pbfPmNWh2cU92NVhEMEk/view?usp= > > > > > sharing > > > > > > > > > > <https://drive.google.com/file/d/0B8l-mZ- > > > pbfPmNWh2cU92NVhEMEk/view?usp= > > > > > sharing> > > > > > > > > > > Thanks. > > > > > > > > > > *Mirage Abeysekara* > > > > > Undergraduate > > > > > Computer Science and Engineering > > > > > University of Moratuwa > > > > > Twitter: https://twitter.com/MiRAGECreator > > > > > GooglePlus: https://plus.google.com/u/0/+MirageAbeysekara > > > > > > > > > > On 12 March 2017 at 00:56, Edward Capriolo <[email protected]> > > > > wrote: > > > > > > > > > > > I added some suggestions/ changes to your document. Looks good on > > my > > > > end. > > > > > > Does anyone else have any suggestions? > > > > > > > > > > > > On Fri, Mar 10, 2017 at 2:31 AM, Mirage Abeysekara < > > > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I prepared a draft design document[1] for the above jira ticket > > > based > > > > > on > > > > > > my > > > > > > > understanding of the project. I would appreciate if you could > > give > > > me > > > > > > some > > > > > > > suggestions or feedback for the design. > > > > > > > > > > > > > > [1] https://docs.google.com/document/d/ > > > 1z1jZIaNqot8SXrelajd1aCBT1q_ > > > > > > > KVHrw5DV2RGJfJtQ/edit > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > *Mirage Abeysekara* > > > > > > > Undergraduate > > > > > > > Computer Science and Engineering > > > > > > > University of Moratuwa > > > > > > > Twitter: https://twitter.com/MiRAGECreator > > > > > > > GooglePlus: https://plus.google.com/u/0/+MirageAbeysekara > > > > > > > > > > > > > > > > > > > > > > > > > > > Mirage, > > > > Sorry I just saw this. I think the important thing is the proposal should > > be interersting to you. Thus I do not want to "assign" things. > > > > I would say the best idea is to look into more data structures: > > http://ieeexplore.ieee.org/document/7328521/ > > > > Or possibly developing some form of lock management: > > http://www.exploredatabase.com/2014/04/distributed-lock- > > manager-concurrency.html > > > > Or building higher level datastructures ontop of CRDTs > > https://ignite.apache.org/features/datastructures.html > > > > To do these we need to solve/handle a couple of issues. > > > > Building something that does a leader election makes sense as well. > > > > the open question in my mind is will we implement something like this: > > > > https://issues.apache.org/jira/browse/VXQUERY-180?jql= > > labels%20%3D%20gsoc2017 > > > > > > Where machine do direct communication vs lazy gossip replication. > > > > Also I have been thinking hard about the per-node vs shared hash maps. I > > thing what we need to there is unify those concepts and make the key of > the > > hashmap be useful for expressing more arbitrary concepts like "only > > replicate this to nodes name xyz" or "only replicate this to nodes in > > datacenter b" or "dont replicate this at all it is only local data" > > > > Having a distributed structure that I can lock is most interesting to me. > > Even if the lock takes 30 seconds to acquire by reaching a quorum of > nodes, > > wondering if we can implement something like this: > > > > https://www.google.com/webhp?sourceid=chrome-instant&ion=1&; > > espv=2&ie=UTF-8#q=bakery+algorithm&* > > > > Anyway there is a lot of ideas there maybe you can take one in a more > > specific direction. > > > https://issues.apache.org/jira/browse/GOSSIP-75 Ok lets take a simple example. Imagine I have some datum like this: { "name":"computePI", "workers": "4" } The question becomes how do we now divide up this work? How can we make sure 4 and only 4 nodes take on this work. One possibility is voting. Node2 starts the process createSharedData( name:computePi/slot1, value: "node2svote:node2" ) now as this passes message passed to node3, node3 will append to the crdt createSharedData( name:computePi/slot1, value: "node2svote:node2" , name:computePi/slot1, value: "node3svote:node2" ) Now, anyone who is interested in this lock/ can keep polling until number of votes > half cluster. But there is other ways to do this. Possibly we use a technique like that to elect a leader who is a decision maker for N seconds, then we elect again. Or maybe this is where that geometric quorum ticket comes in, maybe we do not attempt to gossip this, maybe we pick N nodes and the communicate directly.
