There is no ephemeral drive you will have to use the root disk. Grabbing a random VM in OVH and running a quick df on it you should have about 60GB of free space for the job to use there.
Clark On Tue, Dec 15, 2015, at 02:00 PM, Egor Guz wrote: > Clark, > > > What about ephemeral storage at OVH vms? I see may storage related errors > (see full output below) these days. > Basically it means Docker cannot create storage device at local drive > > -- Logs begin at Mon 2015-12-14 06:40:09 UTC, end at Mon 2015-12-14 > 07:00:38 UTC. -- > Dec 14 06:45:50 > <http://logs.openstack.org/58/251158/3/check/gate-functional-dsvm-magnum-k8s/5ed0e01/logs/bay-nodes/worker-test_replication_controller_apis-172.24.5.11/docker.txt.gz#_Dec_14_06_45_50> > te-egw4i5xthw-0-nmaiwpjhkqg6-kube-minion-5emvszmbwpi2 systemd[1]: Stopped > Docker Application Container Engine. > Dec 14 06:47:54 > <http://logs.openstack.org/58/251158/3/check/gate-functional-dsvm-magnum-k8s/5ed0e01/logs/bay-nodes/worker-test_replication_controller_apis-172.24.5.11/docker.txt.gz#_Dec_14_06_47_54> > te-egw4i5xthw-0-nmaiwpjhkqg6-kube-minion-5emvszmbwpi2 systemd[1]: > Starting Docker Application Container Engine... > Dec 14 06:48:00 > <http://logs.openstack.org/58/251158/3/check/gate-functional-dsvm-magnum-k8s/5ed0e01/logs/bay-nodes/worker-test_replication_controller_apis-172.24.5.11/docker.txt.gz#_Dec_14_06_48_00> > te-egw4i5xthw-0-nmaiwpjhkqg6-kube-minion-5emvszmbwpi2 docker[1022]: > Warning: '-d' is deprecated, it will be removed soon. See usage. > Dec 14 06:48:00 > <http://logs.openstack.org/58/251158/3/check/gate-functional-dsvm-magnum-k8s/5ed0e01/logs/bay-nodes/worker-test_replication_controller_apis-172.24.5.11/docker.txt.gz#_Dec_14_06_48_00> > te-egw4i5xthw-0-nmaiwpjhkqg6-kube-minion-5emvszmbwpi2 docker[1022]: > time="2015-12-14T06:48:00Z" level=warning msg="please use 'docker daemon' > instead." > Dec 14 06:48:03 > <http://logs.openstack.org/58/251158/3/check/gate-functional-dsvm-magnum-k8s/5ed0e01/logs/bay-nodes/worker-test_replication_controller_apis-172.24.5.11/docker.txt.gz#_Dec_14_06_48_03> > te-egw4i5xthw-0-nmaiwpjhkqg6-kube-minion-5emvszmbwpi2 docker[1022]: > time="2015-12-14T06:48:03.447936206Z" level=info msg="Listening for HTTP > on unix (/var/run/docker.sock)" > Dec 14 06:48:06 > <http://logs.openstack.org/58/251158/3/check/gate-functional-dsvm-magnum-k8s/5ed0e01/logs/bay-nodes/worker-test_replication_controller_apis-172.24.5.11/docker.txt.gz#_Dec_14_06_48_06> > te-egw4i5xthw-0-nmaiwpjhkqg6-kube-minion-5emvszmbwpi2 docker[1022]: > time="2015-12-14T06:48:06.280086735Z" level=fatal msg="Error starting > daemon: error initializing graphdriver: Non existing device > docker-docker--pool" > Dec 14 06:48:06 > <http://logs.openstack.org/58/251158/3/check/gate-functional-dsvm-magnum-k8s/5ed0e01/logs/bay-nodes/worker-test_replication_controller_apis-172.24.5.11/docker.txt.gz#_Dec_14_06_48_06> > te-egw4i5xthw-0-nmaiwpjhkqg6-kube-minion-5emvszmbwpi2 systemd[1]: > docker.service: main process exited, code=exited, status=1/FAILURE > Dec 14 06:48:06 > <http://logs.openstack.org/58/251158/3/check/gate-functional-dsvm-magnum-k8s/5ed0e01/logs/bay-nodes/worker-test_replication_controller_apis-172.24.5.11/docker.txt.gz#_Dec_14_06_48_06> > te-egw4i5xthw-0-nmaiwpjhkqg6-kube-minion-5emvszmbwpi2 systemd[1]: Failed > to start Docker Application Container Engine. > Dec 14 06:48:06 > <http://logs.openstack.org/58/251158/3/check/gate-functional-dsvm-magnum-k8s/5ed0e01/logs/bay-nodes/worker-test_replication_controller_apis-172.24.5.11/docker.txt.gz#_Dec_14_06_48_06> > te-egw4i5xthw-0-nmaiwpjhkqg6-kube-minion-5emvszmbwpi2 systemd[1]: Unit > docker.service entered failed state. > Dec 14 06:48:06 > <http://logs.openstack.org/58/251158/3/check/gate-functional-dsvm-magnum-k8s/5ed0e01/logs/bay-nodes/worker-test_replication_controller_apis-172.24.5.11/docker.txt.gz#_Dec_14_06_48_06> > te-egw4i5xthw-0-nmaiwpjhkqg6-kube-minion-5emvszmbwpi2 systemd[1]: > docker.service failed. > > > http://logs.openstack.org/58/251158/3/check/gate-functional-dsvm-magnum-k8s/5ed0e01/logs/bay-nodes/worker-test_replication_controller_apis-172.24.5.11/docker.txt.gz > > > — > Egor > > > > > On 12/13/15, 10:51, "Clark Boylan" <[email protected]> wrote: > > >On Sat, Dec 12, 2015, at 02:16 PM, Hongbin Lu wrote: > >> Hi, > >> > >> As Kai Qiang mentioned, magnum gate recently had a bunch of random > >> failures, which occurred on creating a nova instance with 2G of RAM. > >> According to the error message, it seems that the hypervisor tried to > >> allocate memory to the nova instance but couldn’t find enough free memory > >> in the host. However, by adding a few “nova hypervisor-show XX” before, > >> during, and right after the test, it showed that the host has 6G of free > >> RAM, which is far more than 2G. Here is a snapshot of the output [1]. You > >> can find the full log here [2]. > >If you look at the dstat log > >http://logs.openstack.org/07/244907/5/check/gate-functional-dsvm-magnum-k8s/5305d7a/logs/screen-dstat.txt.gz > >the host has nowhere near 6GB free memory and less than 2GB. I think you > >actually are just running out of memory. > >> > >> Another observation is that most of the failure happened on a node with > >> name “devstack-trusty-ovh-*” (You can verify it by entering a query [3] > >> at http://logstash.openstack.org/ ). It seems that the jobs will be fine > >> if they are allocated to a node other than “ovh”. > >I have just done a quick spot check of the total memory on > >devstack-trusty hosts across HPCloud, Rackspace, and OVH using `free -m` > >and the results are 7480, 7732, and 6976 megabytes respectively. Despite > >using 8GB flavors in each case there is variation and OVH comes in on > >the low end for some reason. I am guessing that you fail here more often > >because the other hosts give you just enough extra memory to boot these > >VMs. > > > >We will have to look into why OVH has less memory despite using flavors > >that should be roughly equivalent. > >> > >> Any hints to debug this issue further? Suggestions are greatly > >> appreciated. > >> > >> [1] http://paste.openstack.org/show/481746/ > >> [2] > >> http://logs.openstack.org/48/256748/1/check/gate-functional-dsvm-magnum-swarm/56d79c3/console.html > >> [3] https://review.openstack.org/#/c/254370/2/queries/1521237.yaml > >> > >> Best regards, > >> Hongbin > > > >Clark > > > >__________________________________________________________________________ > >OpenStack Development Mailing List (not for usage questions) > >Unsubscribe: [email protected]?subject:unsubscribe > >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
