Hi Yi, Please see inline...
On Thu, May 30, 2019 at 5:04 PM Yi Yang (杨燚)-云服务集团 <yangy...@inspur.com> wrote: > Also cc dev mailing list for getting more responses. > > > > *发件人:* Yi Yang (杨燚)-云服务集团 > *发送时间:* 2019年5月30日 14:08 > *收件人:* 'mdsal-...@lists.opendaylight.org' < > mdsal-...@lists.opendaylight.org>; 'controller-dev@lists.opendaylight.org' > <controller-dev@lists.opendaylight.org>; ' > openflowplugin-...@lists.opendaylight.org' < > openflowplugin-...@lists.opendaylight.org> > *抄送:* 'robert.va...@pantheon.tech' <robert.va...@pantheon.tech>; ' > tompante...@gmail.com' <tompante...@gmail.com>; ' > avish...@luminanetworks.com' <avish...@luminanetworks.com>; ' > abhijit.kumbh...@ericsson.com' <abhijit.kumbh...@ericsson.com> > *主题:* Is Read from follower shard ok and openflowplugin master must be > shard leader? > *重要性:* 高 > > > > Hi, folks > > > > I have some questions about ODL clustering and openflowplugin clustering, > look forward to getting your great help, thank you in advance. > > > > # Q1. Is only leader node responsible for synchronizing data store to > other followers for any shard? > > > > # Q2. Openflowplugin clustering also has master, per its document, only > openflowplugin master node can do write operation against inventory data > store, then what if this openflowplugin master node is follower shard? > OpenFlow plugin is driven by the devices connected to it, in the clustered setup. OpenFlow plugin allows you to connect your device to any of the controller node (one or more), and internally it will decide which node from the cluster will be the owner/master of the device using Cluster SIngleton Service + EOS. Once the owner/master is decided, that owner/master is the one allowed to write data to the "operational" inventory (plugin don't write to config inventory). > > > # Q3. Can we do more granular shard per openflow node(DPID) in inventory? > I don’t think it makes sense that the inventory for one openflowplugin > cluster is replicated to all the other openflowplugin clusters (assume > there are many openflowplugin clusters because many south nodes/devices are > there) > are you assuming multiple OpenDaylight cluster instance running and sharing data to each other ? e.g 2 cluster setup running and sharing data throw some external mechanism or a cluster with 6 nodes in it? If you are looking at the scale of 10000 device, and assuming that each cluster can manage 500 devices, you will have to deploy 20 cluster setup or you will have to create cluster with 60 nodes in it. Both of these options are pretty much not practical for the production environment. > > > # Q4. Anybody can recommend node number of a ODL cluster which will manage > 10000 compute/network nodes? I think leader nodes will have too high > workload if number of ODL cluster node is too big so that it can’t do > horizontal scale, per current default shard strategy, every node has all > the data store, that looks more like data store replication, not distribute > data store on all the nodes. > In my experience and opinion, ODL in clustered setup is not a solution here. As i mentioned above, with cluster setup i can think of two possible solution as i mentioned above. Deploying 20 cluster will be operational nightmare (E.g per cluster partition issues, device switching between cluster, device inventory data sharing across cluster on device switching etc). Apart from that you will need external mechanism to share the data between these clusters. And depends on your application, things can get even more complicated to maintain in production environment. If you go with the second option of 60 nodes in cluster, i am not even sure this cluster even will boot up properly :), let alone managing the devices. To make it work, you need to go with the prefix-based-sharding and cook a solution per device (per deivce shard, nodes where this shard can be replicated, making sure that device connection only switch to the node where the devie shard is replicated etc etc etc). > > > # Q5. Is it possible to run an asymmetric ODL cluster? I mean some nodes > are full stack (there are netvirt, sfc, genius, etc), some nodes are > southbound only (only install openflowplugin, ovsdb). I don’t think we must > run other stuff in south bound device management nodes except southbound > protocols. > I think you can do that, but if you want HA for your application and southbound plugins and also you want to run these in exlusion, 3 node cluster is not going to work (atleast you need 4 nodes in cluster). > > > #Q6. I know data store read can be done in any node, but is it read from > local shard in fact? Per document, it seems shard manager is doing this, if > local shard is not leader, it will do this from remote shard leader. > > > > #Q7. Anybody can propose a good ODL clustering solution for a super scale > data center which has 10000 nodes? > In my experience, if you are looking for stable production environment with low operation cost (logistic, resource, support etc), ODL in "clustering" environment is probably not at-par solution. Luis and myself, shared some high level thoughts on how we can achieve this kind of scale using horizontally scalable system in the ONS summit. Here is the deck if you want to get more details. https://docs.google.com/presentation/d/1gDLHyyuh8VVRpzHpTq9GDkv4XKAe3EaSbm2uGJFTiO8/edit?usp=sharing > > > #Q8. Does shard size have any limitation? Per my evaluation from 3 nodes > inventory, inventory for 2000 nodes will be more than 2G, for so big data > set, can IMDS handle it very efficiently? I understand CDS is also using > IMDS locally, right? > > > > #Q9. Is it feasible to use distributed database as data store backend? I > saw opencontrail/Tungsten Fabric is using Cassandra to save all the config > data. It looks like a good idea to use existing database clustering > solution, a big concern is it doesn’t support data change notification and > listener, do we have some other way to do this for such databases? > > > > #Q10. Can we split data store of a module into more shards? I mean these > shards include different data set, they form whole data store. I think this > is a good way to do distributed data store and is more horizontally > scalable. It will be better if every node which hosts data store shard can > read it. > > > > #Q11. An application have one instance in every node, how does ODL decide > which application instance to handle data change notification and data > listener? > > > > #12. How does an application use entity ownership service? What’s the > difference between master, leader and EOS owner? I’m confused a bit. > _______________________________________________ > controller-dev mailing list > controller-dev@lists.opendaylight.org > https://lists.opendaylight.org/mailman/listinfo/controller-dev > -- Thanks Anil
_______________________________________________ controller-dev mailing list controller-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/controller-dev
