Hi,

OVH is a new cloud provider for openstack-infra nodes:
http://www.openstack.org/blog/2015/12/announcing-a-new-cloud-provider-for-openstacks-ci-system-ovh/

It appears that selection of nodes on any cloud provider is a matter of luck: "When a developer uploads a proposed change to an OpenStack project, available instances from any of our contributing cloud providers will be used interchangeably to test it."

You might want to ping people in #openstack-infra to find a point of contact for them (OVH) and/or to work with the infra folks directly to see about troubleshooting this further.


On 12/12/2015 02:16 PM, Hongbin Lu wrote:
Hi,

As Kai Qiang mentioned, magnum gate recently had a bunch of random
failures, which occurred on creating a nova instance with 2G of RAM.
According to the error message, it seems that the hypervisor tried to
allocate memory to the nova instance but couldn’t find enough free
memory in the host. However, by adding a few “nova hypervisor-show XX”
before, during, and right after the test, it showed that the host has 6G
of free RAM, which is far more than 2G. Here is a snapshot of the output
[1]. You can find the full log here [2].

Another observation is that most of the failure happened on a node with
name “devstack-trusty-ovh-*” (You can verify it by entering a query [3]
at http://logstash.openstack.org/ ). It seems that the jobs will be fine
if they are allocated to a node other than “ovh”.

Any hints to debug this issue further? Suggestions are greatly appreciated.

<http://logstash.openstack.org/>

[1] <http://logstash.openstack.org/>http://paste.openstack.org/show/481746/

[2]
http://logs.openstack.org/48/256748/1/check/gate-functional-dsvm-magnum-swarm/56d79c3/console.html

[3] https://review.openstack.org/#/c/254370/2/queries/1521237.yaml

Best regards,

Hongbin

*From:*Kai Qiang Wu [mailto:wk...@cn.ibm.com]
*Sent:* December-09-15 7:23 AM
*To:* openstack-dev@lists.openstack.org
*Subject:* [openstack-dev] [Infra][nova][magnum] Jenkins failed quite
often for "Cannot set up guest memory 'pc.ram': Cannot allocate memory"

Hi All,

I am not sure what changes these days, We found quite often now, the
Jenkins failed for:


http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/libvirt/libvirtd.txt.gz

2015-12-09 08:52:27.892
<http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/libvirt/libvirtd.txt.gz#_2015-12-09_08_52_27_892>+0000:
22957: debug : qemuMonitorJSONCommandWithFd:264 : Send command
'{"execute":"qmp_capabilities","id":"libvirt-1"}' for write with FD -1
2015-12-09 08:52:27.892
<http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/libvirt/libvirtd.txt.gz#_2015-12-09_08_52_27_892>+0000:
22957: debug : qemuMonitorSend:959 : QEMU_MONITOR_SEND_MSG:
mon=0x7fa66400c6f0 msg={"execute":"qmp_capabilities","id":"libvirt-1"}
  fd=-1
2015-12-09 08:52:27.941
<http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/libvirt/libvirtd.txt.gz#_2015-12-09_08_52_27_941>+0000:
22951: debug : virNetlinkEventCallback:347 : dispatching to max 0
clients, called from event watch 6
2015-12-09 08:52:27.941
<http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/libvirt/libvirtd.txt.gz#_2015-12-09_08_52_27_941>+0000:
22951: debug : virNetlinkEventCallback:360 : event not handled.
2015-12-09 08:52:27.941
<http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/libvirt/libvirtd.txt.gz#_2015-12-09_08_52_27_941>+0000:
22951: debug : virNetlinkEventCallback:347 : dispatching to max 0
clients, called from event watch 6
2015-12-09 08:52:27.941
<http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/libvirt/libvirtd.txt.gz#_2015-12-09_08_52_27_941>+0000:
22951: debug : virNetlinkEventCallback:360 : event not handled.
2015-12-09 08:52:27.941
<http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/libvirt/libvirtd.txt.gz#_2015-12-09_08_52_27_941>+0000:
22951: debug : virNetlinkEventCallback:347 : dispatching to max 0
clients, called from event watch 6
2015-12-09 08:52:27.941
<http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/libvirt/libvirtd.txt.gz#_2015-12-09_08_52_27_941>+0000:
22951: debug : virNetlinkEventCallback:360 : event not handled.
2015-12-09 08:52:28.070
<http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/libvirt/libvirtd.txt.gz#_2015-12-09_08_52_28_070>+0000:
22951: error : qemuMonitorIORead:554 : Unable to read from monitor:
Connection reset by peer
2015-12-09 08:52:28.070
<http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/libvirt/libvirtd.txt.gz#_2015-12-09_08_52_28_070>+0000:
22951: error : qemuMonitorIO:690 : internal error: early end of file
from monitor: possible problem:
Cannot set up guest memory 'pc.ram': Cannot allocate memory




It not hit such resource issue before. I am not sure if Infra or nova
guys know about it ?


Thanks

Best Wishes,
--------------------------------------------------------------------------------
Kai Qiang Wu (吴开强 Kennan)
IBM China System and Technology Lab, Beijing

E-mail: wk...@cn.ibm.com <mailto:wk...@cn.ibm.com>
Tel: 86-10-82451647
Address: Building 28(Ring Building), ZhongGuanCun Software Park,
No.8 Dong Bei Wang West Road, Haidian District Beijing P.R.China 100193
--------------------------------------------------------------------------------
Follow your heart. You are miracle!



__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to