Dear Neville,
Could you please share output of "ls -l /var/lib/nova/instances" on both
hosts? The user id's of nova on both is probably different. You can
check /etc/passwd files on both host.
Regards,
Huseyin COTUK
On 21-04-2015 15:43, Neville wrote:
I'm trying to setup live migration in Openstack using Ceph RBD backed
volumes. From what I understand I also need to put the libvirt folder
/var/lib/nova/instances on shared storage for it to work as Nova tests
for this as part of the migration process. I decided to look at using
CephFS for this purpose.
I've created a CephFS file system and mounted it on my two compute
nodes as /var/lib/nova/instances but I'm getting some strange results.
It seems like after one of the hosts accesses a file then it becomes
locked and the other host can't access it. For example, I complete the
following steps:
1\ Create new Openstack instance using the boot from image (create new
volume) option. New instance is created with RBD backed volume as
expected. On the relevant compute host I see the instance folder
created under /var/lib/nova/instances/{/instance id}/ with two files
inside libvirt.xml and console.log. If I cat the libvirt.xml file it
works as expected.
2\ Live migrate the instance to other host. Appears to work as
expected although instance status in Horizon stays as migrating
forever. I can see instance has moved to second host by running virsh
list on both hosts.
4\ Now, if I attempt to cat the libvirt.xml file on the new host I get
"Operation not permitted".
I'm assuming this isn't what's expected?
To test this further I tried the following basic tests:
On Host 2:
root@devops-kvm02:/var/lib/nova/instances
<mailto:root@devops-kvm02:/var/lib/nova/instances># echo hello > test
root@devops-kvm02:/var/lib/nova/instances
<mailto:root@devops-kvm02:/var/lib/nova/instances># cat test
hello
root@devops-kvm02:/var/lib/nova/instances
<mailto:root@devops-kvm02:/var/lib/nova/instances>#
Then from Host 1:
root@devops-kvm01:/var/lib/nova/instances
<mailto:root@devops-kvm01:/var/lib/nova/instances># cat test
cat: test: Operation not permitted
root@devops-kvm01:/var/lib/nova/instances
<mailto:root@devops-kvm01:/var/lib/nova/instances>#
Then back on Host 2:
root@devops-kvm02:/var/lib/nova/instances
<mailto:root@devops-kvm02:/var/lib/nova/instances># cat test
cat: test: Operation not permitted
root@devops-kvm02:/var/lib/nova/instances
<mailto:root@devops-kvm02:/var/lib/nova/instances>#
Should this even work? My understanding is CephFS allows concurrent
access but I'm not sure if there is some file locking going on that I
need to understand.
Thanks,
Neville
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
