Chris and I were talking about the various bugs related to not cleaning
up allocations during failures, of which I've reported two more today
[1][2].
The discovery process here is really just me doing code inspection, then
opening a bug, writing a recreate functional test and then fixing the
bug. This is based on a few other bugs we already fixed (for evacuate,
shelve offload, resize and live migration).
We got to talking about a report we could run at the end of a CI job run
which dumps the allocations out of Placement and then we could use that
to figure out what we are missing.
We actually already do something like this with
nova/tools/hooks/post_test_hook.sh in the nova-next job, as seen here
[3]. That runs the 'nova-manage db archive_deleted_rows' CLI a few times
to make sure things are getting archived properly.
I was thinking we could leverage that post_test_hook script to do
something similar for dumping allocations from Placement.
The devil, of course, is in the details, because we don't have a CLI for
Placement yet. There are some patches for the OSC plugin though [4]. If
we got those in, at least the ones to list resource providers, usages
and allocations, we could get a start on this.
There would likely be other wrinkles, like if there are still instances
lingering when the script runs, we'd have to take those into account. I
think in a failed job, we wouldn't care as much, but in a successful job
where Tempest passes, we'd expect allocations to be cleaned up because
the instances shouldn't be leaking from Tempest.
Anyway, these are just some ideas if someone wanted to start hacking on
this. It should be self-testing since like I said it's in the nova-next
job so if you're just playing around with patches in nova you can see
the results show up in that post_test_hook log.
[1] https://bugs.launchpad.net/nova/+bug/1713786
[2] https://bugs.launchpad.net/nova/+bug/1713796
[3]
http://logs.openstack.org/61/496861/1/check/gate-tempest-dsvm-neutron-nova-next-full-ubuntu-xenial-nv/572ceed/logs/devstack-gate-post_test_hook.txt.gz
[4] https://review.openstack.org/#/q/project:openstack/osc-placement
--
Thanks,
Matt
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
