Thanks for the explanation Robert. When we refer access pattern and shard layout, I assume that it is in context of a given transaction owned by an application - please correct me if I am wrong.
As a detour, the reference to transactions leads to a bunch of questions on how MD-SAL transactions - as we know in context of IMDS, could map on to external backend for a given shard (which CDT enables) For few transaction capabilities which are inherent in IMDS, there may not be equivalent concept/support in an external backend of choice. For example, concept like Transaction Chain. Other implementation-specific aspects of transaction could also vary drastically - for example, a) Snapshotting+MVCC as "I" of ACID is applicable for IMDS - but may not be true in case of Etcd / Cassandra - this mismatch may not have deeper ramifications to the end-user b) strong-consistency for writes may be a do-it-yourself if backend choice is Cassandra whereas its ingrained in IMDS Does this essentially mean that based on choice of backend, some concepts of transactions could be just no-op/ will be thrown with an unsupported exception (eg. transaction-chain) to manage the uniformity of broker contracts? Regards Muthu -----Original Message----- From: Robert Varga [mailto:n...@hq.sk] Sent: Tuesday, December 27, 2016 9:10 PM To: Muthukumaran K <muthukumara...@ericsson.com>; Colin Dixon <co...@colindixon.com>; Shrenik Jain <shrenik.j...@research.iiit.ac.in> Cc: integration-...@lists.opendaylight.org <d...@lists.opendaylight.org>; controller-dev <controller-dev@lists.opendaylight.org>; mdsal-...@lists.opendaylight.org; intern-ad...@opendaylight.org Subject: Re: [controller-dev] Interested in Contribution : "Replacing the MD-SAL data store with etcd" On 12/27/2016 11:38 AM, Muthukumaran K wrote: > Hi Robert, > > Looking at > https://github.com/opendaylight/mdsal/blob/master/dom/mdsal-dom-api/sr > c/main/java/org/opendaylight/mdsal/dom/api/DOMDataTreeShardingService. > java > > Few clarifications: > For same prefix duplicate producers are not allowed - this is > understandable. But there is a possibility that two distinct > producers/shards can have overlapping prefixes - if this is allowed, > would not there be a scenario wherein we could end up with a superset > shard and subset shard > > Eg. Let's assume there is a subtree whose prefix is /a/b/c/d mapped to shard > 1 and another /a/b/c/d/e/f mapped to another shard. > In other words, we can have recursive shards (hypothetical worst case could > be that these recursive shards could have their own backend instances). > Would this not lead to relatively complex read and write routing > implementations? Are such overlaps prevented by Sharding service contracts ? This is allowed and the two shards have a parent/child relationship, where the parent understands that it has a subordinate shard. There can (in theory) be infinite nesting, although I am not sure if the implementation handles more than one level. As for complexity of data access... it is *relatively* complex, but all it really boils down to is a recursive scatter/gather algorithm, where the superior shard delegates parts of the request to its subordinates and merges the responses -- which is pretty much bread-and-butter when dealing with tree-based structures. In any case, while possible, we envision that most applications will only talk to leaf shards, hence scatter/gather will not kick in. The cost of determining which shard is the apex for a producer is paid when the producer is instantiated, hence the steady-state cost should typically be zero. If the application access pattern and sharding layout do not align, you will get sucky performance, similar to what you get if you try to write to multiple module-based shards today. But that is a deployment-time engineering and application co-existence issue, which we cannot solve at the infrastructure layer. Regards, Robert _______________________________________________ controller-dev mailing list controller-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/controller-dev
