Hi Numan, Thanks for the proposal. We have also been thinking about this use-case.
If I’m reading this accurately (and I may not be), it seems that the proposal is to not have any OVN NB (CUD) operations (R operations outside the scope) done by the api_worker threads but rather by a new journal thread. If this is indeed the case, I’d like to consider the scenario when there any N neutron nodes, each node with M worker threads. The journal thread at the each node contain list of pending operations. Could there be (sequence) dependency in the pending operations amongst each the journal threads in the nodes that prevents them from getting applied (for e.g. Logical_Router_Port and Logical_Switch_Port inter-dependency), because we are returning success on neutron operations that have still not been committed to the NB DB. Couple of clarifications and thoughts below. Thanks Amitabha <abis...@us.ibm.com> > On Jul 13, 2016, at 1:20 AM, Numan Siddique <nusid...@redhat.com> wrote: > > Adding the proper tags in subject > > On Wed, Jul 13, 2016 at 1:22 PM, Numan Siddique <nusid...@redhat.com > <mailto:nusid...@redhat.com>> wrote: > Hi Neutrinos, > > Presently, In the OVN ML2 driver we have 2 ways to sync neutron DB and OVN DB > - At neutron-server startup, OVN ML2 driver syncs the neutron DB and OVN DB > if sync mode is set to repair. > - Admin can run the "neutron-ovn-db-sync-util" to sync the DBs. > > Recently, in the v2 of networking-odl ML2 driver (Please see (1) below which > has more details). (ODL folks please correct me if I am wrong here) > > - a journal thread is created which does the CRUD operations of neutron > resources asynchronously (i.e it sends the REST APIs to the ODL controller). Would this be the equivalent of making OVSDB transactions to the OVN NB DB? > - a maintenance thread is created which does some cleanup periodically and > at startup does full sync if it detects ODL controller cold reboot. > > > Few question I have > - can OVN ML2 driver take same or similar approach. Are there any advantages > in taking this approach ? One advantage is neutron resources can be > created/updated/deleted even if the OVN ML2 driver has lost connection to the > ovsdb-server. The journal thread would eventually sync these resources in the > OVN DB. I would like to know the communities thoughts on this. If we can make it work, it would indeed be a huge plus for system wide upgrades and some corner cases in the code (ACL specifically), where the post_commit relies on all transactions to be successful and doesn’t revert the neutron db if something fails. > > - Are there are other ML2 drivers which might have to handle the DB sync's > (cases where the other controllers also maintain their own DBs) and how they > are handling it ? > > - Can a common approach be taken to sync the neutron DB and controller DBs ? > > > ----------------------------------------------------------------------------------------------------------- > > (1) > Sync threads created by networking-odl ML2 driver > -------------------------------------------------- > ODL ML2 driver creates 2 threads (threading.Thread module) at init > - Journal thread > - Maintenance thread > > Journal thread > ---------------- > The journal module creates a new journal table by name “opendaylightjournal” > - > https://github.com/openstack/networking-odl/blob/master/networking_odl/db/models.py#L23 > > <https://github.com/openstack/networking-odl/blob/master/networking_odl/db/models.py#L23> > > Journal thread will be in loop waiting for the sync event from the ODL ML2 > driver. > > - ODL ML2 driver resource (network, subnet, port) precommit functions when > called by the ML2 plugin adds an entry in the “opendaylightjournal” table > with the resource data and sets the journal operation state for this entry to > “PENDING”. > - The corresponding resource postcommit function of the ODL ML2 plugin when > called, sets the sync event flag. > - A timer is also created which sets the sync event flag when it expires > (the default value is 10 seconds). > - Journal thread wakes up, looks into the “opendaylightjournal” table with > the entries with state “pending” and runs the CRUD operation on those > resources in the ODL DB. Once done, it sets the state to “completed”. > > Maintenance thread > ------------------ > Maintenance thread does 3 operations > - JournalCleanup - Delete completed rows from journal table > “opendaylightjournal”. > - CleanupProcessing - Mark orphaned processing rows to pending. > - Full sync - Re-sync when detecting an ODL "cold reboot”. > > > > Thanks > Numan > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
