Re: [ceph-users] Ceph RBD Mirroring

2019-09-14 Thread Oliver Freyermuth
Dear Jason, Am 15.09.19 um 00:03 schrieb Jason Dillaman: > I was able to repeat this issue locally by restarting the primary OSD > for the "rbd_mirroring" object. It seems that a regression was > introduced w/ the introduction of Ceph msgr2 in that upon reconnect, > the connection type for the cli

Re: [ceph-users] Ceph RBD Mirroring

2019-09-14 Thread Jason Dillaman
I was able to repeat this issue locally by restarting the primary OSD for the "rbd_mirroring" object. It seems that a regression was introduced w/ the introduction of Ceph msgr2 in that upon reconnect, the connection type for the client switches from ANY to V2 -- but only for the watcher session an

Re: [ceph-users] Ceph RBD Mirroring

2019-09-13 Thread Oliver Freyermuth
Am 13.09.19 um 18:38 schrieb Jason Dillaman: On Fri, Sep 13, 2019 at 11:30 AM Oliver Freyermuth wrote: Am 13.09.19 um 17:18 schrieb Jason Dillaman: On Fri, Sep 13, 2019 at 10:41 AM Oliver Freyermuth wrote: Am 13.09.19 um 16:30 schrieb Jason Dillaman: On Fri, Sep 13, 2019 at 10:17 AM Jason

Re: [ceph-users] Ceph RBD Mirroring

2019-09-13 Thread Jason Dillaman
On Fri, Sep 13, 2019 at 11:30 AM Oliver Freyermuth wrote: > > Am 13.09.19 um 17:18 schrieb Jason Dillaman: > > On Fri, Sep 13, 2019 at 10:41 AM Oliver Freyermuth > > wrote: > >> > >> Am 13.09.19 um 16:30 schrieb Jason Dillaman: > >>> On Fri, Sep 13, 2019 at 10:17 AM Jason Dillaman > >>> wrote:

Re: [ceph-users] Ceph RBD Mirroring

2019-09-13 Thread Oliver Freyermuth
Am 13.09.19 um 17:18 schrieb Jason Dillaman: On Fri, Sep 13, 2019 at 10:41 AM Oliver Freyermuth wrote: Am 13.09.19 um 16:30 schrieb Jason Dillaman: On Fri, Sep 13, 2019 at 10:17 AM Jason Dillaman wrote: On Fri, Sep 13, 2019 at 10:02 AM Oliver Freyermuth wrote: Dear Jason, thanks for th

Re: [ceph-users] Ceph RBD Mirroring

2019-09-13 Thread Jason Dillaman
On Fri, Sep 13, 2019 at 10:41 AM Oliver Freyermuth wrote: > > Am 13.09.19 um 16:30 schrieb Jason Dillaman: > > On Fri, Sep 13, 2019 at 10:17 AM Jason Dillaman wrote: > >> > >> On Fri, Sep 13, 2019 at 10:02 AM Oliver Freyermuth > >> wrote: > >>> > >>> Dear Jason, > >>> > >>> thanks for the very d

Re: [ceph-users] Ceph RBD Mirroring

2019-09-13 Thread Oliver Freyermuth
Am 13.09.19 um 16:30 schrieb Jason Dillaman: On Fri, Sep 13, 2019 at 10:17 AM Jason Dillaman wrote: On Fri, Sep 13, 2019 at 10:02 AM Oliver Freyermuth wrote: Dear Jason, thanks for the very detailed explanation! This was very instructive. Sadly, the watchers look correct - see details inli

Re: [ceph-users] Ceph RBD Mirroring

2019-09-13 Thread Oliver Freyermuth
Am 13.09.19 um 16:17 schrieb Jason Dillaman: On Fri, Sep 13, 2019 at 10:02 AM Oliver Freyermuth wrote: Dear Jason, thanks for the very detailed explanation! This was very instructive. Sadly, the watchers look correct - see details inline. Am 13.09.19 um 15:02 schrieb Jason Dillaman: On Thu,

Re: [ceph-users] Ceph RBD Mirroring

2019-09-13 Thread Jason Dillaman
On Fri, Sep 13, 2019 at 10:17 AM Jason Dillaman wrote: > > On Fri, Sep 13, 2019 at 10:02 AM Oliver Freyermuth > wrote: > > > > Dear Jason, > > > > thanks for the very detailed explanation! This was very instructive. > > Sadly, the watchers look correct - see details inline. > > > > Am 13.09.19 um

Re: [ceph-users] Ceph RBD Mirroring

2019-09-13 Thread Jason Dillaman
On Fri, Sep 13, 2019 at 10:02 AM Oliver Freyermuth wrote: > > Dear Jason, > > thanks for the very detailed explanation! This was very instructive. > Sadly, the watchers look correct - see details inline. > > Am 13.09.19 um 15:02 schrieb Jason Dillaman: > > On Thu, Sep 12, 2019 at 9:55 PM Oliver Fr

Re: [ceph-users] Ceph RBD Mirroring

2019-09-13 Thread Oliver Freyermuth
Dear Jason, thanks for the very detailed explanation! This was very instructive. Sadly, the watchers look correct - see details inline. Am 13.09.19 um 15:02 schrieb Jason Dillaman: On Thu, Sep 12, 2019 at 9:55 PM Oliver Freyermuth wrote: Dear Jason, thanks for taking care and developing a p

Re: [ceph-users] Ceph RBD Mirroring

2019-09-13 Thread Jason Dillaman
On Thu, Sep 12, 2019 at 9:55 PM Oliver Freyermuth wrote: > > Dear Jason, > > thanks for taking care and developing a patch so quickly! > > I have another strange observation to share. In our test setup, only a single > RBD mirroring daemon is running for 51 images. > It works fine with a constant

Re: [ceph-users] Ceph RBD Mirroring

2019-09-12 Thread Oliver Freyermuth
Dear Jason, thanks for taking care and developing a patch so quickly! I have another strange observation to share. In our test setup, only a single RBD mirroring daemon is running for 51 images. It works fine with a constant stream of 1-2 MB/s, but at some point after roughly 20 hours, _all_

Re: [ceph-users] Ceph RBD Mirroring

2019-09-11 Thread Jason Dillaman
On Wed, Sep 11, 2019 at 12:57 PM Oliver Freyermuth wrote: > > Dear Jason, > > I played a bit more with rbd mirroring and learned that deleting an image at > the source (or disabling journaling on it) immediately moves the image to > trash at the target - > but setting rbd_mirroring_delete_delay

Re: [ceph-users] Ceph RBD Mirroring

2019-09-11 Thread Oliver Freyermuth
Dear Jason, I played a bit more with rbd mirroring and learned that deleting an image at the source (or disabling journaling on it) immediately moves the image to trash at the target - but setting rbd_mirroring_delete_delay helps to have some more grace time to catch human mistakes. However,

Re: [ceph-users] Ceph RBD Mirroring

2019-09-10 Thread Oliver Freyermuth
Dear Jason, On 2019-09-10 23:04, Jason Dillaman wrote: > On Tue, Sep 10, 2019 at 2:08 PM Oliver Freyermuth > wrote: >> >> Dear Jason, >> >> On 2019-09-10 18:50, Jason Dillaman wrote: >>> On Tue, Sep 10, 2019 at 12:25 PM Oliver Freyermuth >>> wrote: Dear Cephalopodians, I have

Re: [ceph-users] Ceph RBD Mirroring

2019-09-10 Thread Jason Dillaman
On Tue, Sep 10, 2019 at 2:08 PM Oliver Freyermuth wrote: > > Dear Jason, > > On 2019-09-10 18:50, Jason Dillaman wrote: > > On Tue, Sep 10, 2019 at 12:25 PM Oliver Freyermuth > > wrote: > >> > >> Dear Cephalopodians, > >> > >> I have two questions about RBD mirroring. > >> > >> 1) I can not get i

Re: [ceph-users] Ceph RBD Mirroring

2019-09-10 Thread Oliver Freyermuth
Dear Jason, On 2019-09-10 18:50, Jason Dillaman wrote: > On Tue, Sep 10, 2019 at 12:25 PM Oliver Freyermuth > wrote: >> >> Dear Cephalopodians, >> >> I have two questions about RBD mirroring. >> >> 1) I can not get it to work - my setup is: >> >> - One cluster holding the live RBD volumes and

Re: [ceph-users] Ceph RBD Mirroring

2019-09-10 Thread Jason Dillaman
On Tue, Sep 10, 2019 at 12:25 PM Oliver Freyermuth wrote: > > Dear Cephalopodians, > > I have two questions about RBD mirroring. > > 1) I can not get it to work - my setup is: > > - One cluster holding the live RBD volumes and snapshots, in pool "rbd", > cluster name "ceph", > running l

[ceph-users] Ceph RBD Mirroring

2019-09-10 Thread Oliver Freyermuth
Dear Cephalopodians, I have two questions about RBD mirroring. 1) I can not get it to work - my setup is: - One cluster holding the live RBD volumes and snapshots, in pool "rbd", cluster name "ceph", running latest Mimic. I ran "rbd mirror pool enable rbd pool" on that cluster and