Re: [ceph-users] RBD image has no active watchers while OpenStack KVM VM is running

2017-12-05 Thread Wido den Hollander
> Op 5 december 2017 om 15:27 schreef Jason Dillaman : > > > On Tue, Dec 5, 2017 at 9:13 AM, Wido den Hollander wrote: > > > >> Op 29 november 2017 om 14:56 schreef Jason Dillaman : > >> > >> > >> We experienced this problem in the past

Re: [ceph-users] RBD image has no active watchers while OpenStack KVM VM is running

2017-12-05 Thread Jason Dillaman
On Tue, Dec 5, 2017 at 9:13 AM, Wido den Hollander wrote: > >> Op 29 november 2017 om 14:56 schreef Jason Dillaman : >> >> >> We experienced this problem in the past on older (pre-Jewel) releases >> where a PG split that affected the RBD header object would

Re: [ceph-users] RBD image has no active watchers while OpenStack KVM VM is running

2017-12-05 Thread Wido den Hollander
> Op 29 november 2017 om 14:56 schreef Jason Dillaman : > > > We experienced this problem in the past on older (pre-Jewel) releases > where a PG split that affected the RBD header object would result in > the watch getting lost by librados. Any chance you know if the >

Re: [ceph-users] RBD image has no active watchers while OpenStack KVM VM is running

2017-11-30 Thread Wido den Hollander
> Op 30 november 2017 om 14:19 schreef Jason Dillaman : > > > On Thu, Nov 30, 2017 at 4:00 AM, Wido den Hollander wrote: > > > >> Op 29 november 2017 om 14:56 schreef Jason Dillaman : > >> > >> > >> We experienced this problem in the

Re: [ceph-users] RBD image has no active watchers while OpenStack KVM VM is running

2017-11-30 Thread Jason Dillaman
On Thu, Nov 30, 2017 at 4:00 AM, Wido den Hollander wrote: > >> Op 29 november 2017 om 14:56 schreef Jason Dillaman : >> >> >> We experienced this problem in the past on older (pre-Jewel) releases >> where a PG split that affected the RBD header object would

Re: [ceph-users] RBD image has no active watchers while OpenStack KVM VM is running

2017-11-30 Thread Wido den Hollander
> Op 29 november 2017 om 14:56 schreef Jason Dillaman : > > > We experienced this problem in the past on older (pre-Jewel) releases > where a PG split that affected the RBD header object would result in > the watch getting lost by librados. Any chance you know if the >

Re: [ceph-users] RBD image has no active watchers while OpenStack KVM VM is running

2017-11-29 Thread Jason Dillaman
We experienced this problem in the past on older (pre-Jewel) releases where a PG split that affected the RBD header object would result in the watch getting lost by librados. Any chance you know if the affected RBD header objects were involved in a PG split? Can you generate a gcore dump of one of

Re: [ceph-users] RBD image has no active watchers while OpenStack KVM VM is running

2017-11-29 Thread Logan Kuhn
We've seen this. Our environment isn't identical though, we use oVirt and connect to ceph (11.2.1) via cinder (9.2.1), but it's so very rare that we've never had any luck in pin pointing it and have a lot less VMs, <300. Regards, Logan - On Nov 29, 2017, at 7:48 AM, Wido den Hollander

[ceph-users] RBD image has no active watchers while OpenStack KVM VM is running

2017-11-29 Thread Wido den Hollander
Hi, On a OpenStack environment I encountered a VM which went into R/O mode after a RBD snapshot was created. Digging into this I found 10s (out of thousands) RBD images which DO have a running VM, but do NOT have a watcher on the RBD image. For example: $ rbd status