Public bug reported: While trying to debug some failures in an Ironic job, there are a lot of inventory update warnings in the n-cpu logs for the placement service:
http://logs.openstack.org/39/404239/14/check/gate-tempest-dsvm-ironic- ipa-wholedisk-pxe_snmp-tinyipa-ubuntu-xenial- nv/8f98498/logs/screen-n-cpu.txt.gz?level=TRACE#_2016-12-21_13_20_25_579 2016-12-21 13:20:25.579 21735 WARNING nova.scheduler.client.report [req- f2ea7aeb-f9b4-4117-a98a-d03cc02e2f3e - -] Failed to update inventory for resource provider 3edc7808-2d06-4598-98df-7d5b890e19e2: 400 400 Bad Request The server could not comply with the request since it is either malformed or otherwise incorrect. Unable to update inventory for resource provider 3edc7808-2d06-4598 -98df-7d5b890e19e2: Invalid inventory for 'VCPU' on resource provider '3edc7808-2d06-4598-98df-7d5b890e19e2'. The reserved value is greater than or equal to total. -- I wanted to see what the body was for that request so thought I could find the matching request by id in the placement-api logs looking for req-f2ea7aeb-f9b4-4117-a98a-d03cc02e2f3e but it's not there: http://logs.openstack.org/39/404239/14/check/gate-tempest-dsvm-ironic- ipa-wholedisk-pxe_snmp-tinyipa-ubuntu-xenial-nv/8f98498/logs/apache /placement-api.txt.gz I guess this is because it's a different context, but it seems we should be able to pass the request id from the nova-compute client into the placement API via a header so we can track this same request in both logs to make debugging easier. -- As a second part of this bug (probably a separate bug), we should log the placement request body somewhere, either in n-cpu or the placement API debug logs. I know that we log the request and response bodies in n-cpu logs when nova is making requests to cinder, for example. That's useful for debugging and could be useful for debugging placement issues too. ** Affects: nova Importance: Undecided Status: New ** Tags: compute placement -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1652099 Title: placement requests from n-cpu logs not found in placement-api logs Status in OpenStack Compute (nova): New Bug description: While trying to debug some failures in an Ironic job, there are a lot of inventory update warnings in the n-cpu logs for the placement service: http://logs.openstack.org/39/404239/14/check/gate-tempest-dsvm-ironic- ipa-wholedisk-pxe_snmp-tinyipa-ubuntu-xenial- nv/8f98498/logs/screen-n-cpu.txt.gz?level=TRACE#_2016-12-21_13_20_25_579 2016-12-21 13:20:25.579 21735 WARNING nova.scheduler.client.report [req-f2ea7aeb-f9b4-4117-a98a-d03cc02e2f3e - -] Failed to update inventory for resource provider 3edc7808-2d06-4598-98df-7d5b890e19e2: 400 400 Bad Request The server could not comply with the request since it is either malformed or otherwise incorrect. Unable to update inventory for resource provider 3edc7808-2d06-4598 -98df-7d5b890e19e2: Invalid inventory for 'VCPU' on resource provider '3edc7808-2d06-4598-98df-7d5b890e19e2'. The reserved value is greater than or equal to total. -- I wanted to see what the body was for that request so thought I could find the matching request by id in the placement-api logs looking for req-f2ea7aeb-f9b4-4117-a98a-d03cc02e2f3e but it's not there: http://logs.openstack.org/39/404239/14/check/gate-tempest-dsvm-ironic- ipa-wholedisk-pxe_snmp-tinyipa-ubuntu-xenial-nv/8f98498/logs/apache /placement-api.txt.gz I guess this is because it's a different context, but it seems we should be able to pass the request id from the nova-compute client into the placement API via a header so we can track this same request in both logs to make debugging easier. -- As a second part of this bug (probably a separate bug), we should log the placement request body somewhere, either in n-cpu or the placement API debug logs. I know that we log the request and response bodies in n-cpu logs when nova is making requests to cinder, for example. That's useful for debugging and could be useful for debugging placement issues too. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1652099/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
