I think the first workaround is the solution we're looking for as it better reflects the fact that opencontrail is a db-less plugin. I hope it will be the easier too, but you can never be too sure with neutron unit tests.
Salvatore On 4 May 2015 at 12:56, Pavel Bondar <pbon...@infoblox.com> wrote: > Hi Kevin, > > Thanks for your answer, that is what I was looking for! > I'll check with you in irc to decide which workaround is better: > 1. Mocking NeutronDbSubnet fetch_subnet for opencontrail tests. > 2. Using session.query() directly in NeutronDbSubnet fetch_subnet. > > - Pavel Bondar > > On 30.04.2015 22:46, Kevin Benton wrote: > > The OpenContrail plugin itself doesn't even use the Neutron DB. I > > believe what you are observing is a side effect of the fake server they > > have for their tests, which does inherit the neutron DB. > > > > When you call a method on the core plugin in the contrail unit test > > case, it will go through their request logic and will be piped into the > > fake server. During this time, the db session that was associated with > > the original context passed to the core plugin will be lost do to its > > conversion to a dict.[1, 2] > > > > So I believe what you're seeing is this. > > > > 1. The FakeServer gets create_port called and starts its transactions. > > 2. It now hits the ipam driver which calls out to the neutron manager to > > get the core plugin handle, which is actually the contrail plugin and > > not the FakeServer. > > 3. IPAM calls _get_subnet on the contrail plugin, which serializes the > > context[1] and sends it to the FakeServer. > > 4. The FakeServer code receives the request and deserializes the > > context[2], which no longer has the db session. > > 5. The FakeServer then ends up starting a new session to read the > > subnet, which will interfere with the transaction you created the port > > under since they are from the same engine. > > > > This is why you can query the DB directly rather than calling the core > > plugin. The good news is that you don't have to worry because the actual > > contrail plugin won't be using any of this logic so you're not actually > > breaking anything. > > > > I think what you'll want to do is add a mock.patch for the > > NeutronDbSubnet fetch_subnet method to monkey patch in a reference to > > their FakeServer's _get_subnet method. Ping me on IRC (kevinbenton) if > > you need help. > > > > 1. > > > https://github.com/openstack/neutron/blob/master/neutron/plugins/opencontrail/contrail_plugin.py#L111 > > 2. > > > https://github.com/openstack/neutron/blob/master/neutron/tests/unit/plugins/opencontrail/test_contrail_plugin.py#L121 > > > > On Thu, Apr 30, 2015 at 6:37 AM, Pavel Bondar <pbon...@infoblox.com > > <mailto:pbon...@infoblox.com>> wrote: > > > > Hi, > > > > I am debugging issue observed in OpenContrail tests[1] and so far it > > does not look obvious. > > > > Issue: > > > > In create_port[2] new transaction is started. > > Port gets created, but disappears right after reading subnet from > plugin > > in reference ipam driver[3]: > > > > > plugin = manager.NeutronManager.get_plugin() > > > return plugin._get_subnet(context, id) > > > > Port no longer seen in transaction, like it never existed before > > (magic?). As a result inserting IPAllocation fails with foreing key > > constraint error: > > > > DBReferenceError: (IntegrityError) FOREIGN KEY constraint failed > > u'INSERT INTO ipallocations (port_id, ip_address, subnet_id, > network_id) > > VALUES (?, ?, ?, ?)' ('aba6eaa2-2b2f-4ab9-97b0-4d8a36659363', > > u'10.0.0.2', u'be7bb05b-d501-4cf3-a29a-3861b3b54950', > > u'169f6a61-b5d0-493a-b7fa-74fd5b445c84') > > }}} > > > > Only OpenContrail tests fail with that error (116 failures[1]). Tests > > for other plugin passes fine. As I see OpenContrail is different from > > other plugins: each call to plugin is wrapped into http request, so > > getting subnet happens in another transaction. In tests > requests.post() > > is mocked and http call gets translated into self.get_subnet(...). > > Stack trace from plugin._get_subnet() to db_base get_subnet() in open > > contrail tests looks next[4]. > > > > Also single test failure with full db debug was uploaded for > > investigation[5]: > > - Port is inserted at 362. > > - Subnet is read by plugin at 384. > > - IPAllocation was tried to be inserted at 407. > > Between Port and IPAllocation insert no COMMIT/ROLLBACK or delete > > statement were issued, so can't find explanation why port no longer > > exists on IPAllocation insert step. > > Am I missing something obvious? > > > > For now I have several workarounds, which are basically do not use > > plugin._get_subnet(). Direct session.query() works without such side > > effects. > > But this issue bothers me much since I can't explain why it even > happens > > in OpenContrail tests. > > Any ideas are welcome! > > > > My best theory for now: OpenContrail silently wipes currently running > > transaction in tests (in this case it doesn't sound good). > > > > Anyone can checkout and debug patch set 50 (where issue is observed) > > from review page[6]. > > > > Thank you in advance. > > > > - Pavel Bondar > > > > [1] > > > http://logs.openstack.org/36/153236/50/check/gate-neutron-python27/dd83d43/testr_results.html.gz > > [2] > > > https://review.openstack.org/#/c/153236/50/neutron/db/db_base_plugin_v2.py > > line 1578 / line 1857 > > [3] > > > https://review.openstack.org/#/c/153236/50/neutron/ipam/drivers/neutrondb_ipam/driver.py > > line 50 > > [4] http://pastebin.com/n0AqhC5x > > [5] http://pastebin.com/BDBAXFy9 > > [6] http://logs.openstack.org/36/153236/ > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > < > http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > > > > > > -- > > Kevin Benton > > > > > > > __________________________________________________________________________ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
