Hi,
On one of my systems I'm having troubles with my RBD storage backend.
At first I thought it was a problem with my code, but after trying the
same code on a second machine I'm a bit confused.
The problem is that the storage backend tries to retrieve the value of a
secret and base64 decode it, that fails.
My debug log shows:
debug : virStorageBackendRBDOpenRADOSConn:65 : Using cephx authorization
debug : virStorageBackendRBDOpenRADOSConn:75 : Looking up secret by
UUID: 322bccea-f2ed-4eae-a7e5-d0793ffb162d
debug : virSecretLookupByUUIDString:14128 : conn=0x7fac9c0009c0,
uuidstr=322bccea-f2ed-4eae-a7e5-d0793ffb162d
debug : virSecretLookupByUUID:14086 : conn=0x7fac9c0009c0,
uuid=322bccea-f2ed-4eae-a7e5-d0793ffb162d
debug : virSecretGetValue:14486 : secret=0x7fac94000d30,
value_size=0x7facad481918, flags=0
debug : virStorageBackendRBDOpenRADOSConn:103 : Found cephx key: `I^%
debug : virStorageBackendRBDOpenRADOSConn:135 : Found 1 RADOS cluster
monitors in the pool configuration
debug : virStorageBackendRBDOpenRADOSConn:159 : RADOS mon_host has been
set to: 31.25.100.131:6789,
error : virStorageBackendRBDOpenRADOSConn:171 : internal error failed to
connect to the RADOS monitor on: 31.25.100.131:6789,
It goes wrong at "Found cephx key: <garbage>"
So I figured it had to be something in my code and I went over the code
again, but nothing seemed odd.
I tried the same checkout (tag: 0.9.13-rc1) on a different host (also
Ubuntu 12.04) and that worked.
The secret and pool XML's are the same, but what I found is that the
secret storage on disk seems to go wrong on one machine.
Notice this behavior:
root@stack01:~# virsh secret-set-value
322bccea-f2ed-4eae-a7e5-d0793ffb162d
AQAE+uJPCFpELBAAkTniQvHabBGj0Quwnu2imA==
Secret value set
root@stack01:~# md5sum
/etc/libvirt/secrets/322bccea-f2ed-4eae-a7e5-d0793ffb162d.base64
b4b147bc522828731f1a016bfa72c073
/etc/libvirt/secrets/322bccea-f2ed-4eae-a7e5-d0793ffb162d.base64
root@stack01:~# virsh secret-set-value
322bccea-f2ed-4eae-a7e5-d0793ffb162d
AQAE+uJPCFpELBAAkTniQvHabBGj0Quwnu2imA==
Secret value set
root@stack01:~# md5sum
/etc/libvirt/secrets/322bccea-f2ed-4eae-a7e5-d0793ffb162d.base64
927e2458c32cc3f6754d91694e41333f
/etc/libvirt/secrets/322bccea-f2ed-4eae-a7e5-d0793ffb162d.base64
root@stack01:~#
As you can see, the md5sum of the file changes when I set the value of
the secret to the same.
I tried the same on the other host:
root@stack02:~# virsh secret-set-value
322bccea-f2ed-4eae-a7e5-d0793ffb162d
AQAE+uJPCFpELBAAkTniQvHabBGj0Quwnu2imA==
Secret value set
root@stack02:~# md5sum
/etc/libvirt/secrets/322bccea-f2ed-4eae-a7e5-d0793ffb162d.base64
c30db27f9ebfe3f7903470d4bd542d1d
/etc/libvirt/secrets/322bccea-f2ed-4eae-a7e5-d0793ffb162d.base64
root@stack02:~# virsh secret-set-value
322bccea-f2ed-4eae-a7e5-d0793ffb162d
AQAE+uJPCFpELBAAkTniQvHabBGj0Quwnu2imA==
Secret value set
root@stack02:~# md5sum
/etc/libvirt/secrets/322bccea-f2ed-4eae-a7e5-d0793ffb162d.base64
c30db27f9ebfe3f7903470d4bd542d1d
/etc/libvirt/secrets/322bccea-f2ed-4eae-a7e5-d0793ffb162d.base64
root@stack02:~# virsh secret-set-value
322bccea-f2ed-4eae-a7e5-d0793ffb162d
AQAE+uJPCFpELBAAkTniQvHabBGj0Quwnu2imA==
Secret value set
root@stack02:~# md5sum
/etc/libvirt/secrets/322bccea-f2ed-4eae-a7e5-d0793ffb162d.base64
c30db27f9ebfe3f7903470d4bd542d1d
/etc/libvirt/secrets/322bccea-f2ed-4eae-a7e5-d0793ffb162d.base64
root@stack02:~#
The md5sum stays the same on stack02.
I verified that stack01 isn't out of disk space or out of inodes, those
are in the acceptable values range.
Any suggestions?
For the record, the hosts:
OS: Ubuntu 12.04 x864_64
Kernel: 3.2.0-25-generic
Libvirt commit: 0fce94fe1bd782ac4c33fdd59d13ee37b3437413
Thank you,
Wido
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
