[Yahoo-eng-team] [Bug 1764622] [NEW] Restarting the web server causes users to get kicked out
Public bug reported: Starting with Django 1.9 users are kicked out to the login screen after the web server is restarted. This is especially severe when running Horizon with a high number of processes. However, if Horizon is running with Django 1.8.19 or older, Horizon can be restarted with little to no impact. Reproduced in Devstack stable/queens using the following additional steps. 1) Configured Apache with 30 processes. > WSGIDaemonProcess horizon user=stack group=stack processes=30 threads=1 > home=/opt/stack/horizon display-name=%{GROUP} 2) Configure Horizon to use Memcached. SESSION_ENGINE = 'django.contrib.sessions.backends.cache' CACHES = { 'default': { 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', 'LOCATION': '127.0.0.1:11211', }, } 3) Log in to Horizon. 4) Restarted Apache. 5) Hit F5 and you will be kicked out to the login screen. Keep hitting F5 or clicking on pages and you will randomly be kicked out back to the login screen. It will keep kicking you out until all processes has been used at least once. ** Affects: horizon Importance: Undecided Status: New ** Description changed: - Starting with Django 1.9 and newer users are kicked out to the login - screen after the web server is restarted. This is especially severe when - running Horizon with a high number of processes. + Starting with Django 1.9 users are kicked out to the login screen after + the web server is restarted. This is especially severe when running + Horizon with a high number of processes. However, if Horizon is running with Django 1.8.19 or older, Horizon can be restarted with little to no impact. Reproduced in Devstack stable/queens using the following additional steps. 1) Configured Apache with 30 processes. > WSGIDaemonProcess horizon user=stack group=stack processes=30 threads=1 home=/opt/stack/horizon display-name=%{GROUP} 2) Configure Horizon to use Memcached. SESSION_ENGINE = 'django.contrib.sessions.backends.cache' CACHES = { - 'default': { - 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', - 'LOCATION': '127.0.0.1:11211', - }, + 'default': { + 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', + 'LOCATION': '127.0.0.1:11211', + }, } 3) Log in to Horizon. 4) Restarted Apache. 5) Hit F5 and you will be kicked out to the login screen. Keep hitting F5 or clicking on pages and you will randomly be kicked out back to the login screen. It will keep kicking you out until all processes has been used at least once. -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1764622 Title: Restarting the web server causes users to get kicked out Status in OpenStack Dashboard (Horizon): New Bug description: Starting with Django 1.9 users are kicked out to the login screen after the web server is restarted. This is especially severe when running Horizon with a high number of processes. However, if Horizon is running with Django 1.8.19 or older, Horizon can be restarted with little to no impact. Reproduced in Devstack stable/queens using the following additional steps. 1) Configured Apache with 30 processes. > WSGIDaemonProcess horizon user=stack group=stack processes=30 threads=1 home=/opt/stack/horizon display-name=%{GROUP} 2) Configure Horizon to use Memcached. SESSION_ENGINE = 'django.contrib.sessions.backends.cache' CACHES = { 'default': { 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', 'LOCATION': '127.0.0.1:11211', }, } 3) Log in to Horizon. 4) Restarted Apache. 5) Hit F5 and you will be kicked out to the login screen. Keep hitting F5 or clicking on pages and you will randomly be kicked out back to the login screen. It will keep kicking you out until all processes has been used at least once. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1764622/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1755000] Re: Enhance nova-specs repo and webpage
** Changed in: nova Status: In Progress => Invalid ** Changed in: nova Assignee: Nguyen Hai (nguyentrihai93) => (unassigned) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1755000 Title: Enhance nova-specs repo and webpage Status in OpenStack Compute (nova): Invalid Bug description: - Change from oslosphinx to openstackdocstheme - Switch to new PTI requirement for building docs - Remove "Template" section from index pages - Clean up To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1755000/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1746706] Re: Navigation needs to be recovered when opening/reloading ngdetail page
Reviewed: https://review.openstack.org/559636 Committed: https://git.openstack.org/cgit/openstack/senlin-dashboard/commit/?id=8929010fc340dbc4d5622a95566bb20c99055b5b Submitter: Zuul Branch:master commit 8929010fc340dbc4d5622a95566bb20c99055b5b Author: Shu MutoDate: Mon Apr 9 15:04:56 2018 +0900 Reproduce navigations Details view does not reproduce side menu and breadcrumb properly by refreshing or linking directory. This patch fixes this issue. Change-Id: I43f673467893f82c6a8ab461488762a28c001399 Closes-Bug: #1746706 ** Changed in: senlin-dashboard Status: In Progress => Fix Released ** Changed in: magnum-ui Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Dashboard (Horizon). https://bugs.launchpad.net/bugs/1746706 Title: Navigation needs to be recovered when opening/reloading ngdetail page Status in OpenStack Dashboard (Horizon): Fix Released Status in Magnum UI: Fix Released Status in senlin-dashboard: Fix Released Status in Zun UI: Fix Released Bug description: Fix for bug 1681627 allows us to reload or directly open Angular-based detail page (ngdetail), but the navigation menu is not recovered correctly and the menu is focused on the first panel (Project -> API access in most cases). This bug is used to track this problem. There are several thing to be considered. * How to know which panel a ngdetail page belongs to. * How to know which dashboard a ngdetail page belongs to. Perhaps most tricky thing is that at now there is a cases where a single ngdetail page is linked from both the project and admin dashboard, but there is no good way to known the dashboard information from URL. In case of reloading it might be recovered from browser history, but it does not work for opening a ngdetail page via a direct URL. We might need to revisit URL of ngdetail page to include a dashboard information as we do for Django panels and Angular Index page. To manage notifications about this bug go to: https://bugs.launchpad.net/horizon/+bug/1746706/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764556] [NEW] "nova list" fails with exception.ServiceNotFound if service is deleted and has no UUID
Public bug reported: We had a testcase where we booted an instance on Newton, migrated it off the compute node, deleted the compute node (and service), upgraded to Pike, created a new compute node with the same name, and migrated the instance back to the compute node. At this point the "nova list" command failed with exception.ServiceNotFound. It appears that since the Service has no UUID the _from_db_object() routine will try to add it, but the service.save() call fails because the service in question has been deleted. I reproduced the issue with stable/pike devstack. I booted an instance, then created a fake entry in the "services" table without a UUID so the table looked like this: mysql> select * from services; +-+-+-++--++---+--+--+-+-+-+-+-+--+ | created_at | updated_at | deleted_at | id | host | binary | topic | report_count | disabled | deleted | disabled_reason | last_seen_up| forced_down | version | uuid | +-+-+-++--++---+--+--+-+-+-+-+-+--+ | 2018-02-20 16:10:07 | 2018-04-16 22:10:46 | NULL| 1 | devstack | nova-conductor | conductor | 477364 |0 | 0 | NULL| 2018-04-16 22:10:46 | 0 | 22 | c041d7cf-5047-4014-b50c-3ba6b5d95097 | | 2018-02-20 16:10:10 | 2018-04-16 22:10:54 | NULL| 2 | devstack | nova-compute | compute | 477149 |0 | 0 | NULL| 2018-04-16 22:10:54 | 0 | 22 | d0cfb63c-8b59-4b65-bb7e-6b89acd3fe35 | | 2018-02-20 16:10:10 | 2018-04-16 20:29:33 | 2018-04-16 20:30:33 | 3 | devstack | nova-compute | compute | 476432 |0 | 3 | NULL| 2018-04-16 20:30:33 | 0 | 22 | NULL | +-+-+-++--++---+--+--+-+-+-+-+-+--+ At this point, running "nova show " worked fine, but running "nova list" failed: stack@devstack:~/devstack$ nova list ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. (HTTP 500) (Request-ID: req-b7e1b5f9-e7b4-4ccf-ba28-e8b3e1acd2f6) The nova-api log looked like this: Apr 16 22:11:00 devstack devstack@n-api.service[4258]: DEBUG nova.compute.api [None req-b7e1b5f9-e7b4-4ccf-ba28-e8b3e1acd2f6 demo demo] Listing 1000 instances in cell 09eb515f-9906-40bf-9be6-63b5e6ee279a(cell1) {{(pid=4261) _get_instances_by_filters_all_cells /opt/stack/nova/nova/compute/api.py:2559}} Apr 16 22:11:00 devstack devstack@n-api.service[4258]: DEBUG oslo_concurrency.lockutils [None req-b7e1b5f9-e7b4-4ccf-ba28-e8b3e1acd2f6 demo demo] Lock "09eb515f-9906-40bf-9be6-63b5e6ee279a" acquired by "nova.context.get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=4261) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:270}} Apr 16 22:11:00 devstack devstack@n-api.service[4258]: DEBUG oslo_concurrency.lockutils [None req-b7e1b5f9-e7b4-4ccf-ba28-e8b3e1acd2f6 demo demo] Lock "09eb515f-9906-40bf-9be6-63b5e6ee279a" released by "nova.context.get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=4261) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:282}} Apr 16 22:11:00 devstack devstack@n-api.service[4258]: DEBUG nova.objects.service [None req-b7e1b5f9-e7b4-4ccf-ba28-e8b3e1acd2f6 demo demo] Generated UUID 4368a7ff-f589-4197-b0b9-d2afdb71ca33 for service 3 {{(pid=4261) _from_db_object /opt/stack/nova/nova/objects/service.py:245}} Apr 16 22:11:00 devstack devstack@n-api.service[4258]: ERROR nova.api.openstack.extensions [None req-b7e1b5f9-e7b4-4ccf-ba28-e8b3e1acd2f6 demo demo] Unexpected exception in API method: ServiceNotFound: Service 3 could not be found. Apr 16 22:11:00 devstack devstack@n-api.service[4258]: ERROR nova.api.openstack.extensions Traceback (most recent call last): Apr 16 22:11:00 devstack devstack@n-api.service[4258]: ERROR nova.api.openstack.extensions File "/opt/stack/nova/nova/api/openstack/extensions.py", line 336, in wrapped Apr 16 22:11:00 devstack devstack@n-api.service[4258]: ERROR nova.api.openstack.extensions return f(*args, **kwargs) Apr 16 22:11:00 devstack devstack@n-api.service[4258]: ERROR nova.api.openstack.extensions File
[Yahoo-eng-team] [Bug 1764530] [NEW] Verify operation in nova
Public bug reported: This bug tracker is for errors with the documentation, use the following as a template and remove or add fields as you see fit. Convert [ ] into [x] to check boxes: - [x] This doc is inaccurate in this way: When say to run the command: nova-status upgrade check it fails unless I run it as a root. But the documentation doesn't say it. I don't know if I make a mistake during the install, or if I should run it as a root. When run as unprivileged user the output is: $ nova-status upgrade check Error: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nova/cmd/status.py", line 459, in main ret = fn(*fn_args, **fn_kwargs) File "/usr/lib/python2.7/dist-packages/nova/cmd/status.py", line 389, in check result = func(self) File "/usr/lib/python2.7/dist-packages/nova/cmd/status.py", line 133, in _check_cellsv2 meta.bind = db_session.get_api_engine() File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 152, in get_api_engine return api_context_manager.get_legacy_facade().get_engine() File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 791, in get_legacy_facade return self._factory.get_legacy_facade() File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 342, in get_legacy_facade self._start() File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 484, in _start engine_args, maker_args) File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 506, in _setup_for_connection "No sql_connection parameter is established") CantStartEngineError: No sql_connection parameter is establishe When run as root the output is: # nova-status upgrade check +---+ | Upgrade Check Results | +---+ | Check: Cells v2 | | Result: Success | | Details: None | +---+ | Check: Placement API | | Result: Success | | Details: None | +---+ | Check: Resource Providers | | Result: Success | | Details: None | +---+ - [ ] This is a doc addition request. - [ ] I have a fix to the document that I can paste below including example: input and output. If you have a troubleshooting or support issue, use the following resources: - Ask OpenStack: http://ask.openstack.org - The mailing list: http://lists.openstack.org - IRC: 'openstack' channel on Freenode --- Release: 17.0.3.dev7 on 2018-04-13 23:03 SHA: fda768b304e05821f7479f9698c59d18bf3d3516 Source: https://git.openstack.org/cgit/openstack/nova/tree/doc/source/install/verify.rst URL: https://docs.openstack.org/nova/queens/install/verify.html ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1764530 Title: Verify operation in nova Status in OpenStack Compute (nova): New Bug description: This bug tracker is for errors with the documentation, use the following as a template and remove or add fields as you see fit. Convert [ ] into [x] to check boxes: - [x] This doc is inaccurate in this way: When say to run the command: nova-status upgrade check it fails unless I run it as a root. But the documentation doesn't say it. I don't know if I make a mistake during the install, or if I should run it as a root. When run as unprivileged user the output is: $ nova-status upgrade check Error: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/nova/cmd/status.py", line 459, in main ret = fn(*fn_args, **fn_kwargs) File "/usr/lib/python2.7/dist-packages/nova/cmd/status.py", line 389, in check result = func(self) File "/usr/lib/python2.7/dist-packages/nova/cmd/status.py", line 133, in _check_cellsv2 meta.bind = db_session.get_api_engine() File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 152, in get_api_engine return api_context_manager.get_legacy_facade().get_engine() File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 791, in get_legacy_facade return self._factory.get_legacy_facade() File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 342, in get_legacy_facade self._start() File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 484, in _start engine_args, maker_args) File "/usr/lib/python2.7/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 506, in _setup_for_connection "No sql_connection parameter is established") CantStartEngineError: No sql_connection parameter is establishe When run as root the
[Yahoo-eng-team] [Bug 1762596] Re: controller nova resize instance dont' work openstack Pike
Looking at your service list output, you only have a single compute service, correct? So you have to resize to the same host, and the placement service is saying that host has no space left for the resize to the new flavor that you're attempting. ** Changed in: nova Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1762596 Title: controller nova resize instance dont' work openstack Pike Status in OpenStack Compute (nova): Invalid Bug description: Openstack Pike OpenSUSE 42.3 with KVM Hypervisor. command exec: nova --debug resize --poll 99624d22-b4e4-464b-98ae-f446ec6acd36 92a7eef9-9ba0-48b9-a30f-be8c835892a7 OUTPUT: DEBUG (session:727) POST call to compute for http://controller.stack.intranet.net:8774/v2.1/servers/99624d22-b4e4-464b-98ae-f446ec6acd36/action used request id req-638be505-68ee-4ded-963c-42d805600973 DEBUG (shell:951) No valid host was found. No valid host found for resize (HTTP 400) (Request-ID: req-638be505-68ee-4ded-963c-42d805600973) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 949, in main OpenStackComputeShell().main(argv) File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 871, in main args.func(self.cs, args) File "/usr/lib/python2.7/site-packages/novaclient/v2/shell.py", line 1864, in do_resize server.resize(flavor, **kwargs) File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 393, in resize return self.manager.resize(self, flavor, **kwargs) File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1566, in resize return self._action('resize', server, info=info, **kwargs) File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1931, in _action info=info, **kwargs) File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1942, in _action_return_resp_and_body return self.api.client.post(url, body=body) File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 294, in post return self.request(url, 'POST', **kwargs) File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 83, in request raise exceptions.from_response(resp, body, url, method) BadRequest: No valid host was found. No valid host found for resize (HTTP 400) (Request-ID: req-638be505-68ee-4ded-963c-42d805600973) ERROR (BadRequest): No valid host was found. No valid host found for resize (HTTP 400) (Request-ID: req-638be505-68ee-4ded-963c-42d805600973) Log service: nova.scheduler.manager => Got no allocation candidates from the Placement API. This may be a temporary occurrence as compute nodes start up and begin reporting inventory to the Placement service. select_destinations ... File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 232, in inner return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 139, in select_destinations raise exception.NoValidHost(reason="") NoValidHost: No valid host was found. : NoValidHost_Remote: No valid host was found. Traceback (most recent call last): Configuration: Enable in Controller and Compute node: allow_migrate_to_same_host = True allow_resize_to_same_host = True nova-status upgrade check => work +---+ | Upgrade Check Results | +---+ | Check: Cells v2 | | Result: Success | | Details: None | +---+ | Check: Placement API | | Result: Success | | Details: None | +---+ | Check: Resource Providers | | Result: Success | | Details: None | +---+ when deploy a instance work don't problem... but didn't resize flavor. the quota is fine, the service also for example nova-placement-api, nova-conductor, nova-compute, nova-scheduler all state up. nova service-list +--+--+-+--+-+---++-+-+ | Id | Binary | Host| Zone | Status | State | Updated_at | Disabled Reason | Forced down | +--+--+-+--+-+---++-+-+ | 19d81123-4c5a-4fc4-9803-bb4a06137ece | nova-conductor | controller1 | internal | enabled | down | 2018-04-07T23:22:14.00 | - | False | | 42b9268b-ff71-4664-ad12-209b4fad6b56 | nova-consoleauth |
[Yahoo-eng-team] [Bug 1713783] Re: After failed evacuation the recovered source compute tries to delete the instance
** No longer affects: nova/newton -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1713783 Title: After failed evacuation the recovered source compute tries to delete the instance Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) ocata series: In Progress Status in OpenStack Compute (nova) pike series: Fix Committed Status in OpenStack Security Advisory: Won't Fix Bug description: Description === In case of a failed evacuation attempt the status of the migration is 'accepted' instead of 'failed' so when source compute is recovered the compute manager tries to delete the instance from the source host. However a secondary fault prevents deleting the allocation in placement so the actual deletion of the instance fails as well. Steps to reproduce == The following functional test reproduces the bug: https://review.openstack.org/#/c/498482/ What it does: initiate evacuation when no valid host is available and evacuation fails, but nova manager still tries to delete the instance. Logs: 2017-08-29 19:11:15,751 ERROR [oslo_messaging.rpc.server] Exception during message handling NoValidHost: No valid host was found. There are not enough hosts available. 2017-08-29 19:11:16,103 INFO [nova.tests.functional.test_servers] Running periodic for compute1 (host1) 2017-08-29 19:11:16,115 INFO [nova.api.openstack.placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/4e8e23ff-0c52-4cf7-8356-d9fa88536316/aggregates" status: 200 len: 18 microversion: 1.1 2017-08-29 19:11:16,120 INFO [nova.api.openstack.placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/4e8e23ff-0c52-4cf7-8356-d9fa88536316/inventories" status: 200 len: 401 microversion: 1.0 2017-08-29 19:11:16,131 INFO [nova.api.openstack.placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/4e8e23ff-0c52-4cf7-8356-d9fa88536316/allocations" status: 200 len: 152 microversion: 1.0 2017-08-29 19:11:16,138 INFO [nova.compute.resource_tracker] Final resource view: name=host1 phys_ram=8192MB used_ram=1024MB phys_disk=1028GB used_disk=1GB total_vcpus=10 used_vcpus=1 pci_stats=[] 2017-08-29 19:11:16,146 INFO [nova.api.openstack.placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/4e8e23ff-0c52-4cf7-8356-d9fa88536316/aggregates" status: 200 len: 18 microversion: 1.1 2017-08-29 19:11:16,151 INFO [nova.api.openstack.placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/4e8e23ff-0c52-4cf7-8356-d9fa88536316/inventories" status: 200 len: 401 microversion: 1.0 2017-08-29 19:11:16,152 INFO [nova.tests.functional.test_servers] Running periodic for compute2 (host2) 2017-08-29 19:11:16,163 INFO [nova.api.openstack.placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/531b1ce8-def1-455d-95b3-4140665d956f/aggregates" status: 200 len: 18 microversion: 1.1 2017-08-29 19:11:16,168 INFO [nova.api.openstack.placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/531b1ce8-def1-455d-95b3-4140665d956f/inventories" status: 200 len: 401 microversion: 1.0 2017-08-29 19:11:16,176 INFO [nova.api.openstack.placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/531b1ce8-def1-455d-95b3-4140665d956f/allocations" status: 200 len: 54 microversion: 1.0 2017-08-29 19:11:16,184 INFO [nova.compute.resource_tracker] Final resource view: name=host2 phys_ram=8192MB used_ram=512MB phys_disk=1028GB used_disk=0GB total_vcpus=10 used_vcpus=0 pci_stats=[] 2017-08-29 19:11:16,192 INFO [nova.api.openstack.placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/531b1ce8-def1-455d-95b3-4140665d956f/aggregates" status: 200 len: 18 microversion: 1.1 2017-08-29 19:11:16,197 INFO [nova.api.openstack.placement.requestlog] 127.0.0.1 "GET /placement/resource_providers/531b1ce8-def1-455d-95b3-4140665d956f/inventories" status: 200 len: 401 microversion: 1.0 2017-08-29 19:11:16,198 INFO [nova.tests.functional.test_servers] Finished with periodics 2017-08-29 19:11:16,255 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/6f70656e737461636b20342065766572/servers/5058200c-478e-4449-88c1-906fdd572662" status: 200 len: 1875 microversion: 2.53 time: 0.056198 2017-08-29 19:11:16,262 INFO [nova.api.openstack.requestlog] 127.0.0.1 "GET /v2.1/6f70656e737461636b20342065766572/os-migrations" status: 200 len: 373 microversion: 2.53 time: 0.004618 2017-08-29 19:11:16,280 INFO [nova.api.openstack.requestlog] 127.0.0.1 "PUT /v2.1/6f70656e737461636b20342065766572/os-services/c269bc74-4720-4de4-a6e5-889080b892a0" status: 200 len: 245 microversion: 2.53 time: 0.016442 2017-08-29 19:11:16,281 INFO [nova.service] Starting compute node (version 16.0.0)
[Yahoo-eng-team] [Bug 1719460] Re: (perf) Unnecessarily joining instance.services when listing instances regardless of microversion
Reviewed: https://review.openstack.org/507854 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a0b4116ed68cb71a9a74fee616b5036f5fda4dd2 Submitter: Zuul Branch:master commit a0b4116ed68cb71a9a74fee616b5036f5fda4dd2 Author: Andrey VolkovDate: Wed Sep 27 09:40:22 2017 +0300 List instances performace optimization Join service table for microversion starting 2.16 only so only include service from 2.16 version. Co-Authored-By: jichenjc Closes-Bug: 1719460 Change-Id: I6c57fa013ee8f6d064fc747906e1234a0aa3e8c2 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1719460 Title: (perf) Unnecessarily joining instance.services when listing instances regardless of microversion Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) pike series: Confirmed Status in OpenStack Compute (nova) queens series: Confirmed Bug description: Microversion 2.16 adds the ability to show the host status of an instance when listing servers with details or showing a single server's details. By default that is only shown for an admin. Change https://review.openstack.org/#/c/38/ helped improve the performance for this by avoiding lazy-loading the instance.services column by doing the join in the DB API when querying the instances from the database. However, that check is not based on version 2.16, like the 2.26 tags check below it. This means that we are unnecessarily joining with the services table when querying instances with microversions < 2.16, which happens, for example, by default in the openstack CLI which uses microversion 2.1. We arguably should make this also conditional on policy so we don't join for non-admins by default, but that's less of an issue probably as non-admins probably aren't listing thousands of instances from the deployment like an admin would. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1719460/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764372] Re: while executing ServerMigrationList use-case an extra call is made to nova-api for fetching server details.
First, this is a python-novaclient bug, not a nova bug. Second, the call to get the server isn't for checking if the server exists, it's because the CLI allows passing in a server name or ID, and if it's a name, we have to look it up to get the ID to pass to the GET /servers/{server_id/migrations API: https://github.com/openstack/python- novaclient/blob/10.1.0/novaclient/v2/shell.py#L3330 So I don't think this is a valid bug. ** Also affects: python-novaclient Importance: Undecided Status: New ** No longer affects: nova ** Changed in: python-novaclient Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1764372 Title: while executing ServerMigrationList use-case an extra call is made to nova-api for fetching server details. Status in python-novaclient: Invalid Bug description: An extra call is made to nova-api for fetching server details to check it's existence. Once existence of server is confirmed, after that "server migration list" call is made to nova-api. In existing nova cli code flow, if server does not exist and "nova server-migration-list {server-id}" is invoked, it will display error to user and will not execute server migration list API call. In contrary, if user directly execute curl command to fetch migration list for server which does not exists, then also same error message is displayed to user. So, it seems extra call to check existence of server. Solution: Code should be improved to eliminate extra nova-api call to check server existence before executing concerned command. Server migration list can directly be executed through curl to avoid extra call to check server existence before invoking server migration list. curl -g -i -X GET http://{hostip}:8774/v2.1/{tenant_id}/servers/{server_id}/migrations -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.23" -H "X-Auth-Token:{token_id}". To manage notifications about this bug go to: https://bugs.launchpad.net/python-novaclient/+bug/1764372/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1583504] Re: The instances which didn't be evacuated will be destroyed when the nova-compute service is restarted.
*** This bug is a duplicate of bug 1713783 *** https://bugs.launchpad.net/bugs/1713783 This was fixed under a different bug: https://review.openstack.org/#/c/499237/ ** This bug has been marked a duplicate of bug 1713783 After failed evacuation the recovered source compute tries to delete the instance -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1583504 Title: The instances which didn't be evacuated will be destroyed when the nova-compute service is restarted. Status in OpenStack Compute (nova): Confirmed Bug description: Description === Normally, if you finished evacuating instance, nova-compute will destroy the existed instances whose state are "done" or "accept" in the migration table from the fault host after the nova-compute service is restarted. However, if the nova-scheduler failed to schedule the hosts, e.g. no valid host, nova-scheduler won't update the migration state of the instance to "failed", so that the nova-compute will destroy this instance after restarting by mistake. Steps to reproduce == 1. Create some instances in the the specific host. 2. Make this host to fault state. 3. Disable nova-compute service in other hosts, aim to mock that nova-scheduler fail to schedule. 4. Recover the fault host and restart nova-compute. 5. Check all instances are still existed in the kvm. Expected result === All instances should be still existed in the kvm. Actual result = All instances are destroyed by nova-compute unfortunately. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1583504/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764460] Re: Cannot hard reboot an instance in error state
** Tags added: libvirt queens-backport-potential ** Changed in: nova Status: New => Confirmed ** Also affects: nova/queens Importance: Undecided Status: New ** Changed in: nova/queens Importance: Undecided => High ** Changed in: nova Importance: Undecided => High ** Changed in: nova/queens Status: New => Confirmed ** Summary changed: - Cannot hard reboot an instance in error state + Cannot hard reboot a libvirt instance in error state (mdev query fails) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1764460 Title: Cannot hard reboot a libvirt instance in error state (mdev query fails) Status in OpenStack Compute (nova): Confirmed Status in OpenStack Compute (nova) queens series: Confirmed Bug description: Nova version: stable/queens fda768b304e05821f7479f9698c59d18bf3d3516 Hypervisor: Libvirt + KVM If an instance doesn't exist in libvirt (failed live migration, compute container rebuilt, etc) a hard reboot or start is no longer able to recreate it. We see this problem occasionally happen for various reasons and in the past a hard reboot would revive the instance. A recent commit is responsible (libvirt: pass the mdevs when rebooting the guest). _get_all_assigned_mediated_devices() throws an instanceNotFound exception when trying to start such an instance. Adding a instance_exists() check solves the issue. --- driver.py.orig 2018-04-16 16:11:42.86972 + +++ driver.py 2018-04-16 16:11:55.901773724 + @@ -5966,6 +5966,8 @@ """ allocated_mdevs = {} if instance: +if not self.instance_exists(instance): +return {} guest = self._host.get_guest(instance) guests = [guest] else: Steps to recreate: 1. Stop an instance 2. Delete the instance-XXX.xml file from /etc/libvirt/qemu/ 3. Start the instance Expected result: instance running Actual result: error: instanceNotFound from nova-compute Logs: 2018-04-16 15:41:09.756 2030272 INFO nova.compute.manager [req-ce2e1036-ab7b-4a98-b343-6ab748326963 32bab887a38f4b6cbcaf83297d4b7812 29e87d21ad14403bb789543e8bc0dab7 - default default] [instance: 0130afdf-f5aa-4ec9-8d0a-71080c70f276] Successfully reverted task state from powering-on on failure for instance. 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server [req-ce2e1036-ab7b-4a98-b343-6ab748326963 32bab887a38f4b6cbcaf83297d4b7812 29e87d21ad14403bb789543e8bc0dab7 - default default] Exception during message handling: InstanceNotFound: Instance 0130afdf-f5aa-4ec9-8d0a-71080c70f276 could not be found. 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server function_name, call_dict, binary) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server self.force_reraise() 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2018-04-16 15:41:09.790 2030272 ERROR
[Yahoo-eng-team] [Bug 1759971] Re: [dvr][fast-exit] a route to a tenant network does not get created in fip namespace if an external network is attached after a tenant network have been attached (race
** Also affects: neutron (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: neutron (Ubuntu Artful) Importance: Undecided Status: New ** Changed in: neutron (Ubuntu Artful) Status: New => Triaged ** Changed in: neutron (Ubuntu Artful) Importance: Undecided => Medium ** Changed in: neutron (Ubuntu Bionic) Importance: Undecided => Medium ** Changed in: neutron (Ubuntu Bionic) Status: New => Triaged ** Also affects: cloud-archive Importance: Undecided Status: New ** Also affects: cloud-archive/pike Importance: Undecided Status: New ** Also affects: cloud-archive/queens Importance: Undecided Status: New ** Changed in: cloud-archive/pike Status: New => Triaged ** Changed in: cloud-archive/pike Importance: Undecided => Medium ** Changed in: cloud-archive/queens Status: New => Triaged ** Changed in: cloud-archive/queens Importance: Undecided => Medium -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1759971 Title: [dvr][fast-exit] a route to a tenant network does not get created in fip namespace if an external network is attached after a tenant network have been attached (race condition) Status in Ubuntu Cloud Archive: Triaged Status in Ubuntu Cloud Archive pike series: Triaged Status in Ubuntu Cloud Archive queens series: Triaged Status in neutron: Fix Released Status in neutron package in Ubuntu: Triaged Status in neutron source package in Artful: Triaged Status in neutron source package in Bionic: Triaged Bug description: Overall, similar scenario to https://bugs.launchpad.net/neutron/+bug/1759956 but a different problem. Relevant agent config options: http://paste.openstack.org/show/718418/ OpenStack Queens from UCA (xenial, GA kernel, deployed via OpenStack charms), 2 external subnets (one routed provider network), 1 tenant subnet, all subnets in the same address scope to trigger "fast exit" logic. Tenant subnet cidr: 192.168.100.0/24 openstack address scope create dev openstack subnet pool create --address-scope dev --pool-prefix 10.232.40.0/21 --pool-prefix 10.232.16.0/21 dev openstack subnet pool create --address-scope dev --pool-prefix 192.168.100.0/24 tenant openstack network create --external --provider-physical-network physnet1 --provider-network-type flat pubnet openstack network segment set --name segment1 d8391bfb-4466-4a45-972c-45ffcec9f6bc openstack network segment create --physical-network physnet2 --network-type flat --network pubnet segment2 openstack subnet create --no-dhcp --subnet-pool dev --subnet-range 10.232.16.0/21 --allocation-pool start=10.232.17.0,end=10.232.17.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment1 pubsubnetl1 openstack subnet create --gateway 10.232.40.100 --no-dhcp --subnet-pool dev --subnet-range 10.232.40.0/21 --allocation-pool start=10.232.41.0,end=10.232.41.255 --dns-nameserver 10.232.36.101 --ip-version 4 --network pubnet --network-segment segment2 pubsubnetl2 openstack network create --internal --provider-network-type vxlan tenantnet openstack subnet create --dhcp --ip-version 4 --subnet-range 192.168.100.0/24 --subnet-pool tenant --dns-nameserver 10.232.36.101 --network tenantnet tenantsubnet # --- # Works in this order when an external network is attached first openstack router create --disable --no-ha --distributed pubrouter openstack router set --disable-snat --external-gateway pubnet --enable pubrouter openstack router add subnet pubrouter tenantsubnet 2018-03-29 23:30:48.933 2050638 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'ne tns', 'exec', 'fip-d0f008fc-dc45-4237-9ce0-a9e1977735eb', 'ip', '-4', 'route', 'replace', '192.168.100.0/24', 'via', '169.254.106.114', 'dev', 'fpr-09fd1 424-7'] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:92 # -- # Doesn't work the other way around - as a fip namespace does not get created before a tenant network is attached openstack router create --disable --no-ha --distributed pubrouter openstack router add subnet pubrouter tenantsubnet openstack router set --disable-snat --external-gateway pubnet --enable pubrouter # to "fix" this we need to re-trigger the right code path openstack router remove subnet pubrouter tenantsubnet openstack router add subnet pubrouter tenantsubnet To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-archive/+bug/1759971/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764489] [NEW] Preallocated disks are deducted twice from disk_available_least when using preallocated_images = space
Public bug reported: Description === When using preallocated file based disks (preallocate_images = space) with the Libvirt virt driver the reported allocation for each disk appears doubled, leaving disk_available_least under reporting the amount of available resources on a compute node. Originally reported and debugged by mschuppert and mdbooth: https://bugzilla.redhat.com/show_bug.cgi?id=1554265 Steps to reproduce == $ crudini --get /etc/nova/nova-cpu.conf DEFAULT preallocate_images space $ nova hypervisor-show eb515aa2-b79e-4f38-a6ea-d493a6e0657f +---+--+ | Property | Value| +---+--+ [..] | disk_available_least | 43 | | free_disk_gb | 48 | [..] | local_gb | 48 | | local_gb_used | 0| [..] +---+--+ $ nova flavor-show 2 ++--+ | Property | Value| ++--+ [..] | disk | 20 | [..] ++--+ $ nova boot --image cirros-0.3.5-x86_64-disk --flavor 2 test [..] $ nova hypervisor-show eb515aa2-b79e-4f38-a6ea-d493a6e0657f +---+--+ | Property | Value| +---+--+ [..] | disk_available_least | 3| | free_disk_gb | 28 | [..] | local_gb | 48 | | local_gb_used | 20 | [..] +---+--+ Expected result === The allocation for each preallocated disk is reported correctly and removed from disk_available_least. Actual result = The allocation for each preallocated disk appears doubled and removed from disk_available_least. Environment === 1. Exact version of OpenStack you are running. See the following list for all releases: http://docs.openstack.org/releases/ fb0b785169e5e422b06e82f2eb58e68f6d2008b3 2. Which hypervisor did you use? (For example: Libvirt + KVM, Libvirt + XEN, Hyper-V, PowerKVM, ...) What's the version of that? Libvirt + KVM 2. Which storage type did you use? (For example: Ceph, LVM, GPFS, ...) What's the version of that? file / qcow2 3. Which networking type did you use? (For example: nova-network, Neutron with OpenVSwitch, ...) N/A Logs & Configs == See above. ** Affects: nova Importance: Undecided Status: New ** Tags: libvirt -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1764489 Title: Preallocated disks are deducted twice from disk_available_least when using preallocated_images = space Status in OpenStack Compute (nova): New Bug description: Description === When using preallocated file based disks (preallocate_images = space) with the Libvirt virt driver the reported allocation for each disk appears doubled, leaving disk_available_least under reporting the amount of available resources on a compute node. Originally reported and debugged by mschuppert and mdbooth: https://bugzilla.redhat.com/show_bug.cgi?id=1554265 Steps to reproduce == $ crudini --get /etc/nova/nova-cpu.conf DEFAULT preallocate_images space $ nova hypervisor-show eb515aa2-b79e-4f38-a6ea-d493a6e0657f +---+--+ | Property | Value| +---+--+ [..] | disk_available_least | 43 | | free_disk_gb | 48 | [..] | local_gb | 48 | | local_gb_used | 0| [..] +---+--+ $ nova flavor-show 2 ++--+ | Property | Value| ++--+ [..] | disk | 20 | [..] ++--+ $ nova boot --image cirros-0.3.5-x86_64-disk
[Yahoo-eng-team] [Bug 1759956] Re: [dvr][fast-exit] incorrect policy rules get deleted when a distributed router has ports on multiple tenant networks
** Also affects: neutron (Ubuntu Bionic) Importance: Undecided Status: Confirmed ** Also affects: neutron (Ubuntu Artful) Importance: Undecided Status: New ** Changed in: neutron (Ubuntu Artful) Status: New => Triaged ** Changed in: neutron (Ubuntu Artful) Importance: Undecided => Medium ** Changed in: neutron (Ubuntu Bionic) Status: Confirmed => Triaged ** Changed in: neutron (Ubuntu Bionic) Importance: Undecided => Medium ** Also affects: cloud-archive Importance: Undecided Status: New ** Also affects: cloud-archive/queens Importance: Undecided Status: New ** Also affects: cloud-archive/pike Importance: Undecided Status: New ** Changed in: cloud-archive/pike Status: New => Triaged ** Changed in: cloud-archive/queens Status: New => Triaged ** Changed in: cloud-archive/pike Importance: Undecided => Medium ** Changed in: cloud-archive/queens Importance: Undecided => Medium -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1759956 Title: [dvr][fast-exit] incorrect policy rules get deleted when a distributed router has ports on multiple tenant networks Status in Ubuntu Cloud Archive: Triaged Status in Ubuntu Cloud Archive pike series: Triaged Status in Ubuntu Cloud Archive queens series: Triaged Status in neutron: Fix Released Status in neutron package in Ubuntu: Triaged Status in neutron source package in Artful: Triaged Status in neutron source package in Bionic: Triaged Bug description: Ubuntu SRU details -- [Impact] See Original Description below. [Test Case] See Original Description below. [Regression Potential] Low. All patches have landed upstream in corresponding stable branches. Original Description TL;DR: ip -4 rule del priority table type unicast will delete the first matching rule it encounters: if there are two rules with the same priority it will just kill the first one it finds. The original setup is described here: https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1759918 OpenStack Queens from UCA (xenial, GA kernel, deployed via OpenStack charms), 2 external subnets (one routed provider network), 2 tenant subnets all in the same address scope to trigger "fast exit". 2 tenant networks attached (subnets 192.168.100.0/24 and 192.168.200.0/24) to a DVR: # 2 rules as expected ip netns exec qrouter-4f9ca9ef-303b-4082-abbc-e50782d9b800 ip rule 0: from all lookup local 32766: from all lookup main 32767: from all lookup default 8: from 192.168.100.0/24 lookup 16 8: from 192.168.200.0/24 lookup 16 # remove 192.168.200.0/24 sometimes deletes an incorrect policy rule openstack router remove subnet pubrouter othertenantsubnet # ip route del contains the cidr 2018-03-29 20:09:52.946 2083594 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'ne tns', 'exec', 'fip-d0f008fc-dc45-4237-9ce0-a9e1977735eb', 'ip', '-4', 'route', 'del', '192.168.200.0/24', 'via', '169.254.93.94', 'dev', 'fpr-4f9ca9ef-3' ] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:92 # ip rule delete is not that specific 2018-03-29 20:09:53.195 2083594 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-4f9ca9ef-303b-4082-abbc-e50782d9b800', 'ip', '-4', 'rule', 'del', 'priority', '8', 'table', '16', 'type', 'unicast'] create_pr ocess /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:92 2018-03-29 20:15:59.210 2083594 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-4f9ca9ef-303b-4082-abbc-e50782d9b800', 'ip', '-4', 'rule', 'show'] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:92 2018-03-29 20:15:59.455 2083594 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-4f9ca9ef-303b-4082-abbc-e50782d9b800', 'ip', '-4', 'rule', 'add', 'from', '192.168.100.0/24', 'priority', '8', 'table', '16', 'type', 'unicast'] create_process /usr/lib/python2.7/dist-packages/neutron/agent/linux/utils.py:92 ip netns exec qrouter-4f9ca9ef-303b-4082-abbc-e50782d9b800 ip rule 0: from all lookup local 32766: from all lookup main 32767: from all lookup default 8: from 192.168.100.0/24 lookup 16 8: from 192.168.200.0/24 lookup 16 # try to delete a rule manually to see what is going on ip netns exec qrouter-4f9ca9ef-303b-4082-abbc-e50782d9b800 ip rule ; ip netns exec qrouter-4f9ca9ef-303b-4082-abbc-e50782d9b800 ip -4 rule del
[Yahoo-eng-team] [Bug 1764481] [NEW] Sometimes dhcp_releasepackets don't reach dnsmasq
Public bug reported: We have seen issues downstream where calling dhcp_release didn't cause the lease to be removed from the leases files used by dnsmasq. There are a couple of scenarios where this could happen: 1. The packet is simply lost, as it is UDP, even though it's being looped-back 2. dnsmasq is reloading, so there is noone to receive it when it arrives For that reason we should make this more robust. A couple of possible solutions are: 1. Send the release more than once in succession. It's probably OK to just send some small number of packets for each lease we want to release, it would easily increase the odds that one makes it through. 2. Check the leases file to make sure dnsmasq processed the release, and re-send for those addresses that were missed. This method is slightly more complicated, but it should also increase the odds, and should do it with fewer packets being generated. Each option has some overhead, but since the option is not releasing the lease, it's worth it. I have proposed option #2 already at https://review.openstack.org/#/c/560703/ but wanted to make sure to get feedback on other proposals that might also solve the issue. ** Affects: neutron Importance: Medium Assignee: Brian Haley (brian-haley) Status: Confirmed ** Tags: l3-ipam-dhcp -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1764481 Title: Sometimes dhcp_releasepackets don't reach dnsmasq Status in neutron: Confirmed Bug description: We have seen issues downstream where calling dhcp_release didn't cause the lease to be removed from the leases files used by dnsmasq. There are a couple of scenarios where this could happen: 1. The packet is simply lost, as it is UDP, even though it's being looped-back 2. dnsmasq is reloading, so there is noone to receive it when it arrives For that reason we should make this more robust. A couple of possible solutions are: 1. Send the release more than once in succession. It's probably OK to just send some small number of packets for each lease we want to release, it would easily increase the odds that one makes it through. 2. Check the leases file to make sure dnsmasq processed the release, and re-send for those addresses that were missed. This method is slightly more complicated, but it should also increase the odds, and should do it with fewer packets being generated. Each option has some overhead, but since the option is not releasing the lease, it's worth it. I have proposed option #2 already at https://review.openstack.org/#/c/560703/ but wanted to make sure to get feedback on other proposals that might also solve the issue. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1764481/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1761405] Re: impossible to disable notifications
** Changed in: nova Assignee: (unassigned) => Matt Riedemann (mriedem) ** Also affects: nova/pike Importance: Undecided Status: New ** Also affects: nova/queens Importance: Undecided Status: New ** Changed in: nova/pike Status: New => Confirmed ** Changed in: nova/pike Importance: Undecided => Low ** Changed in: nova/queens Status: New => Confirmed ** Changed in: nova/queens Importance: Undecided => Low -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1761405 Title: impossible to disable notifications Status in OpenStack Compute (nova): Confirmed Status in OpenStack Compute (nova) pike series: Confirmed Status in OpenStack Compute (nova) queens series: Confirmed Bug description: Hi, As far as I can see from https://github.com/openstack/nova/blob/stable/queens/nova/rpc.py#L60-L86, it's currently (i.e. in queens) not possible to instruct nova to not send notifications - you either get versioned, legacy, or both. In clouds which don't have ceilometer, nothing is consuming the notifications, which keep piling up in rabbitmq : $ sudo rabbitmqctl list_queues -p openstack|grep -v 0$ Listing queues ... notifications.error 32 notifications.info 44908 notifications_designate.info15161 versioned_notifications.error 24 versioned_notifications.info22285 Could we please get a knob to make nova not send any notification ? Thanks ! To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1761405/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764460] [NEW] Cannot hard reboot an instance in error state
Public bug reported: Nova version: stable/queens fda768b304e05821f7479f9698c59d18bf3d3516 Hypervisor: Libvirt + KVM If an instance doesn't exist in libvirt (failed live migration, compute container rebuilt, etc) a hard reboot or start is no longer able to recreate it. We see this problem occasionally happen for various reasons and in the past a hard reboot would revive the instance. A recent commit is responsible (libvirt: pass the mdevs when rebooting the guest). _get_all_assigned_mediated_devices() throws an instanceNotFound exception when trying to start such an instance. Adding a instance_exists() check solves the issue. --- driver.py.orig 2018-04-16 16:11:42.86972 + +++ driver.py 2018-04-16 16:11:55.901773724 + @@ -5966,6 +5966,8 @@ """ allocated_mdevs = {} if instance: +if not self.instance_exists(instance): +return {} guest = self._host.get_guest(instance) guests = [guest] else: Steps to recreate: 1. Stop an instance 2. Delete the instance-XXX.xml file from /etc/libvirt/qemu/ 3. Start the instance Expected result: instance running Actual result: error: instanceNotFound from nova-compute Logs: 2018-04-16 15:41:09.756 2030272 INFO nova.compute.manager [req-ce2e1036-ab7b-4a98-b343-6ab748326963 32bab887a38f4b6cbcaf83297d4b7812 29e87d21ad14403bb789543e8bc0dab7 - default default] [instance: 0130afdf-f5aa-4ec9-8d0a-71080c70f276] Successfully reverted task state from powering-on on failure for instance. 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server [req-ce2e1036-ab7b-4a98-b343-6ab748326963 32bab887a38f4b6cbcaf83297d4b7812 29e87d21ad14403bb789543e8bc0dab7 - default default] Exception during message handling: InstanceNotFound: Instance 0130afdf-f5aa-4ec9-8d0a-71080c70f276 could not be found. 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server Traceback (most recent call last): 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/exception_wrapper.py", line 76, in wrapped 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server function_name, call_dict, binary) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server self.force_reraise() 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/exception_wrapper.py", line 67, in wrapped 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/compute/manager.py", line 186, in decorated_function 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server "Error: %s", e, instance=instance) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__ 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server self.force_reraise() 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) 2018-04-16 15:41:09.790 2030272 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/compute/manager.py", line 156,
[Yahoo-eng-team] [Bug 1764392] [NEW] Avoid bandwidth usage db query in notifications when the virt driver does not support collecting such data
Public bug reported: The notify_usage_exist() function always query the DB to get the bandwidth usage data. But such data is only available if the CONF.bandwidth_poll_interval is set to a positive number (600 is the default) and the virt driver supports such data collection. Today only xenapi driver supports this data collection. So it would make sense to avoid the DB query when it is known that such data is not available. ** Affects: nova Importance: Low Status: New ** Tags: notifications ** Changed in: nova Importance: Undecided => Low ** Tags added: notifications -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1764392 Title: Avoid bandwidth usage db query in notifications when the virt driver does not support collecting such data Status in OpenStack Compute (nova): New Bug description: The notify_usage_exist() function always query the DB to get the bandwidth usage data. But such data is only available if the CONF.bandwidth_poll_interval is set to a positive number (600 is the default) and the virt driver supports such data collection. Today only xenapi driver supports this data collection. So it would make sense to avoid the DB query when it is known that such data is not available. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1764392/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764390] [NEW] Replace passing system_metadata to notification functions with instance.system_metadata usage
Public bug reported: Both notify_usage_exists() [1] and info_from_instance() [2] functions used in the notification code path get the system_metadata passed in. Instead we should use the instance.system_metadata directly whenever it is possible. [1] https://github.com/openstack/nova/blob/57d3b7093259b625672a98b0ff41643175f6cb82/nova/compute/utils.py#L278 [2]https://github.com/openstack/nova/blob/57d3b7093259b625672a98b0ff41643175f6cb82/nova/notifications/base.py#L382 ** Affects: nova Importance: Low Status: New ** Tags: notifications ** Changed in: nova Importance: Undecided => Low ** Tags added: notifications -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1764390 Title: Replace passing system_metadata to notification functions with instance.system_metadata usage Status in OpenStack Compute (nova): New Bug description: Both notify_usage_exists() [1] and info_from_instance() [2] functions used in the notification code path get the system_metadata passed in. Instead we should use the instance.system_metadata directly whenever it is possible. [1] https://github.com/openstack/nova/blob/57d3b7093259b625672a98b0ff41643175f6cb82/nova/compute/utils.py#L278 [2]https://github.com/openstack/nova/blob/57d3b7093259b625672a98b0ff41643175f6cb82/nova/notifications/base.py#L382 To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1764390/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764385] [NEW] no intimation to the admin that nova-api is stopped during execution of polling compute
Public bug reported: Since polling_compute is a background process there are no intimation to the admin that nova-api service is stopped during execution of polling compute. Only by checking /var/log/messages logs, one can detect the error. however, admin must be informed about failure of the service. There must be a mechanism (e.g.: email etc.) to notify admin that nova-api service is down. ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1764385 Title: no intimation to the admin that nova-api is stopped during execution of polling compute Status in OpenStack Compute (nova): New Bug description: Since polling_compute is a background process there are no intimation to the admin that nova-api service is stopped during execution of polling compute. Only by checking /var/log/messages logs, one can detect the error. however, admin must be informed about failure of the service. There must be a mechanism (e.g.: email etc.) to notify admin that nova-api service is down. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1764385/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764372] [NEW] while executing ServerMigrationList use-case an extra call is made to nova-api for fetching server details.
Public bug reported: An extra call is made to nova-api for fetching server details to check it's existence. Once existence of server is confirmed, after that "server migration list" call is made to nova-api. In existing nova cli code flow, if server does not exist and "nova server-migration-list {server-id}" is invoked, it will display error to user and will not execute server migration list API call. In contrary, if user directly execute curl command to fetch migration list for server which does not exists, then also same error message is displayed to user. So, it seems extra call to check existence of server. Solution: Code should be improved to eliminate extra nova-api call to check server existence before executing concerned command. Server migration list can directly be executed through curl to avoid extra call to check server existence before invoking server migration list. curl -g -i -X GET http://{hostip}:8774/v2.1/{tenant_id}/servers/{server_id}/migrations -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.23" -H "X-Auth-Token:{token_id}". ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1764372 Title: while executing ServerMigrationList use-case an extra call is made to nova-api for fetching server details. Status in OpenStack Compute (nova): New Bug description: An extra call is made to nova-api for fetching server details to check it's existence. Once existence of server is confirmed, after that "server migration list" call is made to nova-api. In existing nova cli code flow, if server does not exist and "nova server-migration-list {server-id}" is invoked, it will display error to user and will not execute server migration list API call. In contrary, if user directly execute curl command to fetch migration list for server which does not exists, then also same error message is displayed to user. So, it seems extra call to check existence of server. Solution: Code should be improved to eliminate extra nova-api call to check server existence before executing concerned command. Server migration list can directly be executed through curl to avoid extra call to check server existence before invoking server migration list. curl -g -i -X GET http://{hostip}:8774/v2.1/{tenant_id}/servers/{server_id}/migrations -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.23" -H "X-Auth-Token:{token_id}". To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1764372/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764330] [NEW] Cannot set --no-share on shared network covered also by "access_as_shared" RBAC policy
Public bug reported: There is no possibility to set network as not shared if it was also shared via RBAC policy for some specific tenant. How to reproduce bug: 1. Create 2 projects (tenants): tenantA and tenantB 2. TenantA creates an external network (ext_net_A) + subnet 3. For the external network neutron automatically creates a wildcard 'access_as_external' RBAC rule 4. TenantA can create a new port on ext_net_A; TenantB is not allowed to do the same 5. Create a new 'access_as_shared' RBAC rule granting TenantB access to ext_net_A 6. TenantB is now able to create a port on ext_net_A 7. TenantA sets the shared flag to True on ext_net_A (openstack network set --share ), which creates a new wildcard 'access_as_shared' RBAC rule 8. TenantA tries to unshare ext_net_A (openstack network set --no-share ), which fails with: HttpException: Conflict There were no ports added or any other changes made to ext_net_A between sharing and unsharing it. Neutron should be able to unshare the network since the only tenant using it (tenantB) is already covered by a specific RBAC rule created in step 5. ** Affects: neutron Importance: Medium Assignee: Slawek Kaplonski (slaweq) Status: Confirmed ** Tags: api queens-backport-potential -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1764330 Title: Cannot set --no-share on shared network covered also by "access_as_shared" RBAC policy Status in neutron: Confirmed Bug description: There is no possibility to set network as not shared if it was also shared via RBAC policy for some specific tenant. How to reproduce bug: 1. Create 2 projects (tenants): tenantA and tenantB 2. TenantA creates an external network (ext_net_A) + subnet 3. For the external network neutron automatically creates a wildcard 'access_as_external' RBAC rule 4. TenantA can create a new port on ext_net_A; TenantB is not allowed to do the same 5. Create a new 'access_as_shared' RBAC rule granting TenantB access to ext_net_A 6. TenantB is now able to create a port on ext_net_A 7. TenantA sets the shared flag to True on ext_net_A (openstack network set --share ), which creates a new wildcard 'access_as_shared' RBAC rule 8. TenantA tries to unshare ext_net_A (openstack network set --no-share ), which fails with: HttpException: Conflict There were no ports added or any other changes made to ext_net_A between sharing and unsharing it. Neutron should be able to unshare the network since the only tenant using it (tenantB) is already covered by a specific RBAC rule created in step 5. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1764330/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764332] [NEW] Unnecessarily use of a list and extra loop call to offload the shelved instances whose shelved time passed the shelved offload time
Public bug reported: Unnecessarily use of a list and extra loop call to offload the shelved instances whose shelved time passed the shelved offload time. To offload the shelved instances whose shelved time passed the shelved offload time, first list is created and then this list of shelved instances is processed in extra loop. Instead of creating a list of offload ready shelved instances and processing them later, same can be done in one loop. Code should be refactored. Instead of creating a list of offload ready shelved instances and processing them later, same can be done in one loop. Instances can be offloaded the time when their shelved time is checked. (Reference File: nova/compute/manger.py, Function: _poll_shelved_instances, Line No: 5847) ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1764332 Title: Unnecessarily use of a list and extra loop call to offload the shelved instances whose shelved time passed the shelved offload time Status in OpenStack Compute (nova): New Bug description: Unnecessarily use of a list and extra loop call to offload the shelved instances whose shelved time passed the shelved offload time. To offload the shelved instances whose shelved time passed the shelved offload time, first list is created and then this list of shelved instances is processed in extra loop. Instead of creating a list of offload ready shelved instances and processing them later, same can be done in one loop. Code should be refactored. Instead of creating a list of offload ready shelved instances and processing them later, same can be done in one loop. Instances can be offloaded the time when their shelved time is checked. (Reference File: nova/compute/manger.py, Function: _poll_shelved_instances, Line No: 5847) To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1764332/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764328] [NEW] Twice call for retrieving flavor details when executing delete flavor use-case
Public bug reported: For each delete flavor request, we are retrieving the flavor information twice (i.e. get complete flavor list and then get flavor by uuid), The second call to get flavor by uuid is not required as first call provides all the necessary information. Code Change is required to eliminate second calls (i.e. get flavor by uuid) . ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1764328 Title: Twice call for retrieving flavor details when executing delete flavor use-case Status in OpenStack Compute (nova): New Bug description: For each delete flavor request, we are retrieving the flavor information twice (i.e. get complete flavor list and then get flavor by uuid), The second call to get flavor by uuid is not required as first call provides all the necessary information. Code Change is required to eliminate second calls (i.e. get flavor by uuid) . To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1764328/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764259] Re: neutron openstack client returns ' Unknown error' instead of the real error
You are right. The fix will probably need to be in the python- openstackclient code. ** Project changed: neutron => python-openstackclient -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1764259 Title: neutron openstack client returns ' Unknown error' instead of the real error Status in python-openstackclient: Incomplete Bug description: For several neutron create actions, when called via the openstack client you do not get the real error issued by the plugin, as you do with the neutronclient. instead yo get: BadRequestException: Unknown error For example, try to create a subnet without a cidr: 1) with the neutron client you see the real error: neutron subnet-create --name sub1 net1 neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. Bad subnets request: a subnetpool must be specified in the absence of a cidr. Neutron server returns request_ids: ['req-8ee84525-6e98-4774-9392-ab8b596cde1a'] 2) with the openstack client the information is missing: openstack subnet create --network net1 sub1 BadRequestException: Unknown error To manage notifications about this bug go to: https://bugs.launchpad.net/python-openstackclient/+bug/1764259/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764282] [NEW] Mutilple times retrieve information of project_id in Delete domain
Public bug reported: In Delete domain usecase, Redundant SQL queries are getting executed, which can lead to performance delay. In Delete domain use case, select query is executed mutilple times to retreive information of project_id. This must be reduced to enhance the performance. Code change is required for handling the redundant SQL queries. There is a need to change the code in keystone/token/backends.sql.py so that the extra queries will be removed. ** Affects: keystone Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1764282 Title: Mutilple times retrieve information of project_id in Delete domain Status in OpenStack Identity (keystone): New Bug description: In Delete domain usecase, Redundant SQL queries are getting executed, which can lead to performance delay. In Delete domain use case, select query is executed mutilple times to retreive information of project_id. This must be reduced to enhance the performance. Code change is required for handling the redundant SQL queries. There is a need to change the code in keystone/token/backends.sql.py so that the extra queries will be removed. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1764282/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764274] [NEW] An extra GET request is executed to fetch tenant information
Public bug reported: In Create user operation an extra GET request is executed to fetch tenant information. tenant information can not be acquired by tenant_name as API is not supported directly for tenant_name. The first GET request is executed with tenant_name then initially 404 error is returned. Id need to be given to get tenant details. So in next GET request the Tenant_ID information is fetched. Here the first GET request is an overhead. ** Affects: keystone Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1764274 Title: An extra GET request is executed to fetch tenant information Status in OpenStack Identity (keystone): New Bug description: In Create user operation an extra GET request is executed to fetch tenant information. tenant information can not be acquired by tenant_name as API is not supported directly for tenant_name. The first GET request is executed with tenant_name then initially 404 error is returned. Id need to be given to get tenant details. So in next GET request the Tenant_ID information is fetched. Here the first GET request is an overhead. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1764274/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764272] [NEW] no records logged which show that httpd.service is dead
Public bug reported: There are no records logged which show that httpd.service is dead. The error is logged in /var/log/messages, this can make it difficult for the administrator to trace the error because /var/log/messages contains the general purpose error logs for overall system. As keystone service runs inside httpd, the non-availability of httpd can make the entire openstack environment stop working. To avoid this issue, there can be two possible solutions: Log the error message in component level log. Here the error message should be logged in keystone.log so that monitoring remains within keystone component. In this case, changes should be done in keystone client end. High level monitoring mechanism such as email containing the error message can also be sent to admin so that monitoring can be made more convenient. ** Affects: keystone Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Identity (keystone). https://bugs.launchpad.net/bugs/1764272 Title: no records logged which show that httpd.service is dead Status in OpenStack Identity (keystone): New Bug description: There are no records logged which show that httpd.service is dead. The error is logged in /var/log/messages, this can make it difficult for the administrator to trace the error because /var/log/messages contains the general purpose error logs for overall system. As keystone service runs inside httpd, the non-availability of httpd can make the entire openstack environment stop working. To avoid this issue, there can be two possible solutions: Log the error message in component level log. Here the error message should be logged in keystone.log so that monitoring remains within keystone component. In this case, changes should be done in keystone client end. High level monitoring mechanism such as email containing the error message can also be sent to admin so that monitoring can be made more convenient. To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/1764272/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1764264] [NEW] bionic cloud-init 18.2 refuses Juju's 'runcmd' stanza
Public bug reported: I haven't quite figured out what is wrong, but I tried bootstrapping bionic with Juju 2.3.6 (proposed) today. I had been successfully bootstrapping on LXD bionic as of last week. This was my first attempt to bootstrap on a MAAS image of bionic. The cloud init version reported in /var/log/cloud-init-output.log is listed as 18.2 (It may be that it has been "successfully" bootstrapping, but this error has been in the logs and we just didn't notice it.) Cloud-init v. 18.2 running 'modules:config' at Mon, 16 Apr 2018 05:51:08 +. Up 28.17 seconds. 2018-04-16 05:51:08,730 - util.py[WARNING]: Running module apt-configure () failed 2018-04-16 05:51:08,930 - schema.py[WARNING]: Invalid config: runcmd: ['set -xe', "mkdir -p '/var/lib/juju'\ncat > '/var/lib/juju/MAASmachine.txt' << 'EOF'\n'hostname: nuc7\n'\nEOF\nchmod 0755 '/var/lib/juju/MAASmachine.txt'", 'set -xe', "install -D -m 644 /dev/null '/etc/systemd/system/juju-clean-shutdown.service'", "printf '%s\\n' '\n[Unit]\nDescription=Stop all network interfaces on shutdown\nDefaultDependencies=false\nAfter=final.target\n\n[Service]\nType=oneshot\nExecStart=/sbin/ifdown -a -v --force\nStandardOutput=tty\nStandardError=tty\n\n[Install]\nWantedBy=final.target\n' > '/etc/systemd/system/juju-clean-shutdown.service'", "/bin/systemctl enable '/etc/systemd/system/juju-clean-shutdown.service'", "install -D -m 644 /dev/null '/var/lib/juju/nonce.txt'", "printf '%s\\n' 'user-admin:bootstrap' > '/var/lib/juju/nonce.txt'"] has non-unique elements I wasn't able to easily figure out what the schema is for cloud-init, as it seems to be read from a file. I imagine it is available somewhere. I don't know if we're doing something wrong, or if the schema is incorrectly stating that "runcmd" cannot have the same bit of text twice. I'm guessing its complaining because we pass "set -xe" in 2 different places? ** Affects: cloud-init Importance: Undecided Status: New ** Affects: juju Importance: Undecided Status: New ** Affects: juju/2.3 Importance: High Status: Triaged ** Tags: bionic juju ** Also affects: juju Importance: Undecided Status: New ** Also affects: juju/2.3 Importance: Undecided Status: New ** Changed in: juju/2.3 Milestone: None => 2.3.6 -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to cloud-init. https://bugs.launchpad.net/bugs/1764264 Title: bionic cloud-init 18.2 refuses Juju's 'runcmd' stanza Status in cloud-init: New Status in juju: New Status in juju 2.3 series: Triaged Bug description: I haven't quite figured out what is wrong, but I tried bootstrapping bionic with Juju 2.3.6 (proposed) today. I had been successfully bootstrapping on LXD bionic as of last week. This was my first attempt to bootstrap on a MAAS image of bionic. The cloud init version reported in /var/log/cloud-init-output.log is listed as 18.2 (It may be that it has been "successfully" bootstrapping, but this error has been in the logs and we just didn't notice it.) Cloud-init v. 18.2 running 'modules:config' at Mon, 16 Apr 2018 05:51:08 +. Up 28.17 seconds. 2018-04-16 05:51:08,730 - util.py[WARNING]: Running module apt-configure () failed 2018-04-16 05:51:08,930 - schema.py[WARNING]: Invalid config: runcmd: ['set -xe', "mkdir -p '/var/lib/juju'\ncat > '/var/lib/juju/MAASmachine.txt' << 'EOF'\n'hostname: nuc7\n'\nEOF\nchmod 0755 '/var/lib/juju/MAASmachine.txt'", 'set -xe', "install -D -m 644 /dev/null '/etc/systemd/system/juju-clean-shutdown.service'", "printf '%s\\n' '\n[Unit]\nDescription=Stop all network interfaces on shutdown\nDefaultDependencies=false\nAfter=final.target\n\n[Service]\nType=oneshot\nExecStart=/sbin/ifdown -a -v --force\nStandardOutput=tty\nStandardError=tty\n\n[Install]\nWantedBy=final.target\n' > '/etc/systemd/system/juju-clean-shutdown.service'", "/bin/systemctl enable '/etc/systemd/system/juju-clean-shutdown.service'", "install -D -m 644 /dev/null '/var/lib/juju/nonce.txt'", "printf '%s\\n' 'user-admin:bootstrap' > '/var/lib/juju/nonce.txt'"] has non-unique elements I wasn't able to easily figure out what the schema is for cloud-init, as it seems to be read from a file. I imagine it is available somewhere. I don't know if we're doing something wrong, or if the schema is incorrectly stating that "runcmd" cannot have the same bit of text twice. I'm guessing its complaining because we pass "set -xe" in 2 different places? To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1764264/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp