Hello, I did not do anything special that I know of. I was just exporting an image from Openstack. We have recently upgraded from Jewel 10.2.3 to Luminous 12.2.2.
Caps for admin: client.admin key: CENSORED auid: 0 caps: [mgr] allow * caps: [mon] allow * caps: [osd] allow * Caps for Cinder: client.cinder key: CENSORED caps: [mgr] allow r caps: [mon] profile rbd, allow command "osd blacklist" caps: [osd] profile rbd pool=vms, profile rbd pool=volumes, profile rbd pool=images Caps for MGR: mgr.0 key: CENSORED caps: [mon] allow * I believe this is causing the virtual machines we have running to crash. Any advice would be appreciated. Please let me know if I need to provide any other details. Thank you, Cary -Dynamic On Mon, Jan 29, 2018 at 7:53 PM, Gregory Farnum <gfar...@redhat.com> wrote: > On Fri, Jan 26, 2018 at 12:14 PM Cary <dynamic.c...@gmail.com> wrote: >> >> Hello, >> >> We are running Luminous 12.2.2. 6 OSD hosts with 12 1TB OSDs, and 64GB >> RAM. Each host has a SSD for Bluestore's block.wal and block.db. >> There are 5 monitor nodes as well with 32GB RAM. All servers have >> Gentoo with kernel, 4.12.12-gentoo. >> >> When I export an image using: >> rbd export pool-name/volume-name /location/image-name.raw >> >> Message similar to below are displayed. The signature check fails >> randomly. And sometimes a message about a bad authorizer, but not >> everytime. >> The image is still exported successfully. >> >> 2018-01-24 17:35:15.616080 7fc8d4024700 0 cephx: >> verify_authorizer_reply bad nonce got 4552544084014661633 expected >> 4552499520046621785 sent 4552499520046621784 >> 2018-01-24 17:35:15.616098 7fc8d4024700 0 -- >> 172.21.32.16:0/1412094654 >> 172.21.32.6:6802/6219 conn(0x7fc8b0078a50 >> :-1 s=STATE_CONNECTING_WAIT_CONNECT_REPLY_AUTH pgs=0 cs=0 >> l=1)._process_connection failed verifying authorize reply >> 2018-01-24 17:35:15.699004 7fc8d4024700 0 SIGN: MSG 2 Message >> signature does not match contents. >> 2018-01-24 17:35:15.699020 7fc8d4024700 0 SIGN: MSG 2Signature on >> message: >> 2018-01-24 17:35:15.699021 7fc8d4024700 0 SIGN: MSG 2 sig: >> 8189090775647585001 >> 2018-01-24 17:35:15.699047 7fc8d4024700 0 SIGN: MSG 2Locally >> calculated signature: >> 2018-01-24 17:35:15.699048 7fc8d4024700 0 SIGN: MSG 2 >> sig_check:140500325643792 >> 2018-01-24 17:35:15.699049 7fc8d4024700 0 Signature failed. >> 2018-01-24 17:35:15.699050 7fc8d4024700 0 -- >> 172.21.32.16:0/1412094654 >> 172.21.32.2:6807/153106 >> conn(0x7fc8bc020870 :-1 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH >> pgs=26018 cs=1 l=1).process Signature check failed >> >> Does anyone know what could cause this, and what I can do to fix it. > > > That's in the cephx authentication code and it's indicating that the secure > signature sent with the message isn't what the local node thinks it should > be. That's pretty odd (a bit flip or something that could actually change it > ought to trigger the messaging checksums directly) and I'm not quite sure > how it could happen. > > But, as you've noticed, it retries and apparently succeeds. How did you > notice this? > -Greg > >> >> >> Thank you, >> >> Cary >> -Dynamic >> _______________________________________________ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com