[ceph-users] CEPHADM_FAILED_SET_OPTION

[arnoud]

Hi,

After having set up RadosGW with keystone authentication the cluster shows this 
warning:

# ceph health detail
HEALTH_WARN Failed to set 1 option(s)
[WRN] CEPHADM_FAILED_SET_OPTION: Failed to set 1 option(s)
   Failed to set rgw.fra option rgw_keystone_implicit_tenants: config set 
failed: error parsing value: 'True' is not one of the permitted values: false, 
true, swift, s3, both, 0, 1, none retval: -22

I might have made a typo but that has been corrected.

# ceph config dump |grep rgw_keystone_implicit_tenants
client.rgw.fra.controller1.lushdcadvanced  
rgw_keystone_implicit_tenants  true 
  *
client.rgw.fra.controller1.pznmufadvanced  
rgw_keystone_implicit_tenants  true 
  *
client.rgw.fra.controller1.tdrqotadvanced  
rgw_keystone_implicit_tenants  true 
  *
client.rgw.fra.controller2.ndxaetadvanced  
rgw_keystone_implicit_tenants  true 
  *
client.rgw.fra.controller2.rodqxhadvanced  
rgw_keystone_implicit_tenants  true 
  *
client.rgw.fra.controller2.wyhjukadvanced  
rgw_keystone_implicit_tenants  true 
  *
client.rgw.fra.controller3.boasgradvanced  
rgw_keystone_implicit_tenants  true 
  *
client.rgw.fra.controller3.lkczbladvanced  
rgw_keystone_implicit_tenants  true 
  *
client.rgw.fra.controller3.qxcteeadvanced  
rgw_keystone_implicit_tenants  true 
  *

# ceph --version
ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5) quincy (stable)

Any idea on how to fix this issue?


Thanks,
Arnoud.


--
Arnoud de Jonge
DevOps Engineer

FUGA | CLOUD
https://fuga.cloud 
arnoud@fuga.cloud 

Phone +31727513408 
Registration ID 64988767
VAT ID NL855935984B01
Twitter @fugacloud

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Per minor-version view on docs.ceph.com

[Robert Sander]

Hi,

On 7/12/23 05:44, Satoru Takeuchi wrote:


I have a request about docs.ceph.com. Could you provide per minor-version views
on docs.ceph.com?


I would like to second that. Sometimes the behaviour of Ceph changes a 
lot between point releases. If the documentation gets unreliable it does 
not shine a good light on the project.


Regards
--
Robert Sander
Heinlein Consulting GmbH
Schwedter Str. 8/9b, 10119 Berlin

https://www.heinlein-support.de

Tel: 030 / 405051-43
Fax: 030 / 405051-19

Amtsgericht Berlin-Charlottenburg - HRB 220009 B
Geschäftsführer: Peer Heinlein - Sitz: Berlin
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: CEPHADM_FAILED_SET_OPTION

[Adam King]

Do you have a `config` section in your RGW spec? That health warning is
from cephadm trying to set options from a spec section like that. There's a
short bit about it at the top of
https://docs.ceph.com/en/latest/cephadm/services/#service-specification.

On Thu, Jul 13, 2023 at 3:39 AM  wrote:

> Hi,
>
> After having set up RadosGW with keystone authentication the cluster shows
> this warning:
>
> # ceph health detail
> HEALTH_WARN Failed to set 1 option(s)
> [WRN] CEPHADM_FAILED_SET_OPTION: Failed to set 1 option(s)
>Failed to set rgw.fra option rgw_keystone_implicit_tenants: config set
> failed: error parsing value: 'True' is not one of the permitted values:
> false, true, swift, s3, both, 0, 1, none retval: -22
>
> I might have made a typo but that has been corrected.
>
> # ceph config dump |grep rgw_keystone_implicit_tenants
> client.rgw.fra.controller1.lushdcadvanced
> rgw_keystone_implicit_tenants  true
>*
> client.rgw.fra.controller1.pznmufadvanced
> rgw_keystone_implicit_tenants  true
>*
> client.rgw.fra.controller1.tdrqotadvanced
> rgw_keystone_implicit_tenants  true
>*
> client.rgw.fra.controller2.ndxaetadvanced
> rgw_keystone_implicit_tenants  true
>*
> client.rgw.fra.controller2.rodqxhadvanced
> rgw_keystone_implicit_tenants  true
>*
> client.rgw.fra.controller2.wyhjukadvanced
> rgw_keystone_implicit_tenants  true
>*
> client.rgw.fra.controller3.boasgradvanced
> rgw_keystone_implicit_tenants  true
>*
> client.rgw.fra.controller3.lkczbladvanced
> rgw_keystone_implicit_tenants  true
>*
> client.rgw.fra.controller3.qxcteeadvanced
> rgw_keystone_implicit_tenants  true
>*
>
> # ceph --version
> ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5) quincy
> (stable)
>
> Any idea on how to fix this issue?
>
>
> Thanks,
> Arnoud.
>
>
> --
> Arnoud de Jonge
> DevOps Engineer
>
> FUGA | CLOUD
> https://fuga.cloud 
> arnoud@fuga.cloud 
>
> Phone +31727513408 
> Registration ID 64988767
> VAT ID NL855935984B01
> Twitter @fugacloud
>
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: CEPHADM_FAILED_SET_OPTION

[arnoud]

Hi Adam,

That section is indeed pretty short. I’m not sure what the difference is 
between the config and the spec section. Most of the settings I put under 
config give an error when I put then under spec.

I have this spec now.

service_type: rgw
service_id: fra
placement:
  label: rgw
  count_per_host: 3
networks:
- 10.103.4.0/24
config:
  rgw enable usage log: true
  rgw keystone accepted roles: Member, _member_, admin
  rgw keystone admin domain: default
  rgw keystone admin password: secret
  rgw keystone admin project: service
  rgw keystone admin user: swift
  rgw keystone api version: 3
  rgw keystone implicit tenants: true
  rgw keystone url: http://10.103.4.254:35357
  rgw s3 auth use keystone: true
  rgw swift account in url: true
  rgw usage log flush threshold: 1024
  rgw usage log tick interval: 30
  rgw usage max shards: 32
  rgw usage max user shards: 1
spec:
  rgw_frontend_port: 8100

I deleted the 'rgw keystone implicit tenants’ settings now, and the warning 
disappeared. Seems like it has been deprecated? The warning message is very 
misleading.


Thanks,
Arnoud.


> On 13 Jul 2023, at 14:07, Adam King  wrote:
> 
> Do you have a `config` section in your RGW spec? That health warning is from 
> cephadm trying to set options from a spec section like that. There's a short 
> bit about it at the top of 
> https://docs.ceph.com/en/latest/cephadm/services/#service-specification.
> 
> On Thu, Jul 13, 2023 at 3:39 AM  > wrote:
>> Hi,
>> 
>> After having set up RadosGW with keystone authentication the cluster shows 
>> this warning:
>> 
>> # ceph health detail
>> HEALTH_WARN Failed to set 1 option(s)
>> [WRN] CEPHADM_FAILED_SET_OPTION: Failed to set 1 option(s)
>>Failed to set rgw.fra option rgw_keystone_implicit_tenants: config set 
>> failed: error parsing value: 'True' is not one of the permitted values: 
>> false, true, swift, s3, both, 0, 1, none retval: -22
>> 
>> I might have made a typo but that has been corrected.
>> 
>> # ceph config dump |grep rgw_keystone_implicit_tenants
>> client.rgw.fra.controller1.lushdcadvanced  
>> rgw_keystone_implicit_tenants  true  
>>  *
>> client.rgw.fra.controller1.pznmufadvanced  
>> rgw_keystone_implicit_tenants  true  
>>  *
>> client.rgw.fra.controller1.tdrqotadvanced  
>> rgw_keystone_implicit_tenants  true  
>>  *
>> client.rgw.fra.controller2.ndxaetadvanced  
>> rgw_keystone_implicit_tenants  true  
>>  *
>> client.rgw.fra.controller2.rodqxhadvanced  
>> rgw_keystone_implicit_tenants  true  
>>  *
>> client.rgw.fra.controller2.wyhjukadvanced  
>> rgw_keystone_implicit_tenants  true  
>>  *
>> client.rgw.fra.controller3.boasgradvanced  
>> rgw_keystone_implicit_tenants  true  
>>  *
>> client.rgw.fra.controller3.lkczbladvanced  
>> rgw_keystone_implicit_tenants  true  
>>  *
>> client.rgw.fra.controller3.qxcteeadvanced  
>> rgw_keystone_implicit_tenants  true  
>>  *
>> 
>> # ceph --version
>> ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5) quincy 
>> (stable)
>> 
>> Any idea on how to fix this issue?
>> 
>> 
>> Thanks,
>> Arnoud.
>> 
>> 
>> --
>> Arnoud de Jonge
>> DevOps Engineer
>> 
>> FUGA | CLOUD
>> https://fuga.cloud  
>> arnoud@fuga.cloud  > >
>> 
>> Phone +31727513408 
>> Registration ID 64988767
>> VAT ID NL855935984B01
>> Twitter @fugacloud
>> 
>> ___
>> ceph-users mailing list -- ceph-users@ceph.io 
>> To unsubscribe send an email to ceph-users-le...@ceph.io 
>> 
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: CEPHADM_FAILED_SET_OPTION

[Adam King]

The `config` section tells cephadm to try to set the given config options.
So it will try something equivalent to "ceph config set rgw.fra
rgw_keystone_implicit_tenants true" and what it reported in the health
warning "'True' is not one of the permitted values: false, true, swift, s3,
both, 0, 1, none" is the error message it got back when it attempted that.
The error message makes me think the option probably isn't deprecated, but
just didn't like the value that was being passed.

On Thu, Jul 13, 2023 at 9:21 AM  wrote:

> Hi Adam,
>
> That section is indeed pretty short. I’m not sure what the difference is
> between the config and the spec section. Most of the settings I put under
> config give an error when I put then under spec.
>
> I have this spec now.
>
> service_type: rgw
> service_id: fra
> placement:
>   label: rgw
>   count_per_host: 3
> networks:
> - 10.103.4.0/24
> config:
>   rgw enable usage log: true
>   rgw keystone accepted roles: Member, _member_, admin
>   rgw keystone admin domain: default
>   rgw keystone admin password: secret
>   rgw keystone admin project: service
>   rgw keystone admin user: swift
>   rgw keystone api version: 3
>   rgw keystone implicit tenants: true
>   rgw keystone url: http://10.103.4.254:35357
>   rgw s3 auth use keystone: true
>   rgw swift account in url: true
>   rgw usage log flush threshold: 1024
>   rgw usage log tick interval: 30
>   rgw usage max shards: 32
>   rgw usage max user shards: 1
> spec:
>   rgw_frontend_port: 8100
>
> I deleted the 'rgw keystone implicit tenants’ settings now, and the
> warning disappeared. Seems like it has been deprecated? The warning message
> is very misleading.
>
>
> Thanks,
> Arnoud.
>
>
> On 13 Jul 2023, at 14:07, Adam King  wrote:
>
> Do you have a `config` section in your RGW spec? That health warning is
> from cephadm trying to set options from a spec section like that. There's a
> short bit about it at the top of
> https://docs.ceph.com/en/latest/cephadm/services/#service-specification.
>
> On Thu, Jul 13, 2023 at 3:39 AM  wrote:
>
>> Hi,
>>
>> After having set up RadosGW with keystone authentication the cluster
>> shows this warning:
>>
>> # ceph health detail
>> HEALTH_WARN Failed to set 1 option(s)
>> [WRN] CEPHADM_FAILED_SET_OPTION: Failed to set 1 option(s)
>>Failed to set rgw.fra option rgw_keystone_implicit_tenants: config set
>> failed: error parsing value: 'True' is not one of the permitted values:
>> false, true, swift, s3, both, 0, 1, none retval: -22
>>
>> I might have made a typo but that has been corrected.
>>
>> # ceph config dump |grep rgw_keystone_implicit_tenants
>> client.rgw.fra.controller1.lushdcadvanced
>> rgw_keystone_implicit_tenants  true
>>*
>> client.rgw.fra.controller1.pznmufadvanced
>> rgw_keystone_implicit_tenants  true
>>*
>> client.rgw.fra.controller1.tdrqotadvanced
>> rgw_keystone_implicit_tenants  true
>>*
>> client.rgw.fra.controller2.ndxaetadvanced
>> rgw_keystone_implicit_tenants  true
>>*
>> client.rgw.fra.controller2.rodqxhadvanced
>> rgw_keystone_implicit_tenants  true
>>*
>> client.rgw.fra.controller2.wyhjukadvanced
>> rgw_keystone_implicit_tenants  true
>>*
>> client.rgw.fra.controller3.boasgradvanced
>> rgw_keystone_implicit_tenants  true
>>*
>> client.rgw.fra.controller3.lkczbladvanced
>> rgw_keystone_implicit_tenants  true
>>*
>> client.rgw.fra.controller3.qxcteeadvanced
>> rgw_keystone_implicit_tenants  true
>>*
>>
>> # ceph --version
>> ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5) quincy
>> (stable)
>>
>> Any idea on how to fix this issue?
>>
>>
>> Thanks,
>> Arnoud.
>>
>>
>> --
>> Arnoud de Jonge
>> DevOps Engineer
>>
>> FUGA | CLOUD
>> https://fuga.cloud 
>> arnoud@fuga.cloud 
>>
>> Phone +31727513408 >
>> Registration ID 64988767
>> VAT ID NL855935984B01
>> Twitter @fugacloud
>>
>> ___
>> ceph-users mailing list -- ceph-users@ceph.io
>> To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: CEPHADM_FAILED_SET_OPTION

[arnoud]

That is weird. As I had set it to ’true’ (lowercase) which is in the set of 
permitted values. I have set it to 1 now, and it is accepted.


Thanks,
Arnoud.

> On 13 Jul 2023, at 15:32, Adam King  wrote:
> 
> The `config` section tells cephadm to try to set the given config options. So 
> it will try something equivalent to "ceph config set rgw.fra 
> rgw_keystone_implicit_tenants true" and what it reported in the health 
> warning "'True' is not one of the permitted values: false, true, swift, s3, 
> both, 0, 1, none" is the error message it got back when it attempted that. 
> The error message makes me think the option probably isn't deprecated, but 
> just didn't like the value that was being passed.
> 
> On Thu, Jul 13, 2023 at 9:21 AM  wrote:
>> Hi Adam,
>> 
>> That section is indeed pretty short. I’m not sure what the difference is 
>> between the config and the spec section. Most of the settings I put under 
>> config give an error when I put then under spec.
>> 
>> I have this spec now.
>> 
>> service_type: rgw
>> service_id: fra
>> placement:
>>   label: rgw
>>   count_per_host: 3
>> networks:
>> - 10.103.4.0/24 
>> config:
>>   rgw enable usage log: true
>>   rgw keystone accepted roles: Member, _member_, admin
>>   rgw keystone admin domain: default
>>   rgw keystone admin password: secret
>>   rgw keystone admin project: service
>>   rgw keystone admin user: swift
>>   rgw keystone api version: 3
>>   rgw keystone implicit tenants: true
>>   rgw keystone url: http://10.103.4.254:35357 
>>   rgw s3 auth use keystone: true
>>   rgw swift account in url: true
>>   rgw usage log flush threshold: 1024
>>   rgw usage log tick interval: 30
>>   rgw usage max shards: 32
>>   rgw usage max user shards: 1
>> spec:
>>   rgw_frontend_port: 8100
>> 
>> I deleted the 'rgw keystone implicit tenants’ settings now, and the warning 
>> disappeared. Seems like it has been deprecated? The warning message is very 
>> misleading.
>> 
>> 
>> Thanks,
>> Arnoud.
>> 
>> 
>>> On 13 Jul 2023, at 14:07, Adam King >> > wrote:
>>> 
>>> Do you have a `config` section in your RGW spec? That health warning is 
>>> from cephadm trying to set options from a spec section like that. There's a 
>>> short bit about it at the top of 
>>> https://docs.ceph.com/en/latest/cephadm/services/#service-specification.
>>> 
>>> On Thu, Jul 13, 2023 at 3:39 AM >> > wrote:
 Hi,
 
 After having set up RadosGW with keystone authentication the cluster shows 
 this warning:
 
 # ceph health detail
 HEALTH_WARN Failed to set 1 option(s)
 [WRN] CEPHADM_FAILED_SET_OPTION: Failed to set 1 option(s)
Failed to set rgw.fra option rgw_keystone_implicit_tenants: config set 
 failed: error parsing value: 'True' is not one of the permitted values: 
 false, true, swift, s3, both, 0, 1, none retval: -22
 
 I might have made a typo but that has been corrected.
 
 # ceph config dump |grep rgw_keystone_implicit_tenants
 client.rgw.fra.controller1.lushdcadvanced  
 rgw_keystone_implicit_tenants  true
*
 client.rgw.fra.controller1.pznmufadvanced  
 rgw_keystone_implicit_tenants  true
*
 client.rgw.fra.controller1.tdrqotadvanced  
 rgw_keystone_implicit_tenants  true
*
 client.rgw.fra.controller2.ndxaetadvanced  
 rgw_keystone_implicit_tenants  true
*
 client.rgw.fra.controller2.rodqxhadvanced  
 rgw_keystone_implicit_tenants  true
*
 client.rgw.fra.controller2.wyhjukadvanced  
 rgw_keystone_implicit_tenants  true
*
 client.rgw.fra.controller3.boasgradvanced  
 rgw_keystone_implicit_tenants  true
*
 client.rgw.fra.controller3.lkczbladvanced  
 rgw_keystone_implicit_tenants  true
*
 client.rgw.fra.controller3.qxcteeadvanced  
 rgw_keystone_implicit_tenants  true
*
 
 # ceph --version
 ceph version 17.2.6 (d7ff0d10654d2280e08f1ab989c7cdf3064446a5) quincy 
 (stable)
>>>

[ceph-users] Re: Cluster down after network outage

[Frank Schilder]

Hi all,

trying to answer Dan and Stefan in one go.

The OSDs were not booting (understood as the ceph-osd daemon starting up), they 
were running all the time. If they actually boot, they come up quite fast even 
though they are HDD and collocated. The time to be marked up is usually 30-60s 
after start, completely fine with me. We are not affected by pglog-dup 
accumulation or other issues causing excessive boot time and memory consumption.

On the affected part of the cluster we have 972 OSDs running on 900 disks, 48 
SSDs with 1 or 4 OSDs depending on size and 852 HDDs with 1 OSD. All OSDs were 
up and running, but network to their hosts was down for a few hours. After 
network came up, a peering-recover-remap frenzy broke loose and we observed the 
death spiral described in this old post: https://narkive.com/KAzvjjPc.4

Specifically, just setting nodown didn't help, I guess our cluster is simply 
too large. I really had to stop recovery as well (norebalance had no effect). I 
had to do a bit of guessing when all OSDs are both, marked up and seen up at 
the same time (not trivial when the nodown flag is present), but after all OSDs 
were seen up for a while and the initial peering was over, enabling recovery 
now worked as expected and didn't lead to further OSDs being seen as down again.

In this particular situation, when the flag nodown is set, it would be 
extremely helpful to have an extra piece of info in the OSD part of ceph status 
showing the number "seen up" so that it becomes easier to be sure that an OSD 
that was marked up some time ago is not "seen down" but not marked down again 
due to the flag. This was really critical for getting the cluster back.

During all this, no parts of the hardware were saturated. However, the way ceph 
handles such a simultaneous OSD mass-return-to-life event seems not very 
clever. As soon as some PGs manage to peer into an active state, they start 
with recovery irrespective of whether or not they are remappings and the 
missing OSDs are actually there and waiting to be marked up. This in turn 
increases the load on the daemons unnecessarily. On top of this 
impatience-implied load amplification comes that the operations scheduler is 
not good at pushing high-priority messages like MON heart beats through to busy 
daemons.

As a consequence, one has a constant up-and down marking of OSDs even though 
the OSDs are actually running fine and nothing crashed. This implies constant 
re-peering of all PGs and accumulation of additional pglog history for any 
successful recovery or remapped write.

The strategy in such a such a situation would actually be to wait for every 
single OSD to be marked up, start peering and only then look if something needs 
recovery. Trying to do this all at the same time is just a mess.

In this particular case, setting osd_recovery_delay_start to a high value looks 
very promising to help dealing with such a situation in a bit more hands-free 
way - if this flag is doing something in octopus. I can see it available in the 
ceph config options on the command line but not in the docs (its implemented 
but not documented). Thanks for bringing this to my attention! Looking at the 
time it took for all OSDs to come up and peering to complete, 10-30 minutes 
might do the job for our cluster. 10-30 minutes on an HDD pool after 
simultaneously marking 900 OSDs up is actually a good time. I assume that 
recovery for a PG will be postponed whenever this PG (or one of its OSDs) has a 
peering event?

When looking at the sequence of events, I would actually also like an 
osd_up_peer_delay_start to postpone peering a bit in case several OSDs are 
starting up to avoid redundant re-peering. The ideal recovery sequence after a 
mass-down of OSDs really seems to be to do each of these as atomic steps:

- get all OSDs up
- peer all PGs
- start recovery if needed

Being able to delay peering after an OSD comes up would give some space for the 
"I'm up" messages to arrive with priority at the MONs. Then the PGs have time 
to say "I'm active+clean" and only then will the trash be taken out.

Thanks for replying to my messages and for pointing me to 
osd_recovery_delay_start.

Best regards,
=
Frank Schilder
AIT Risø Campus
Bygning 109, rum S14


From: Dan van der Ster 
Sent: Wednesday, July 12, 2023 6:58 PM
To: Frank Schilder
Cc: ceph-users@ceph.io
Subject: Re: [ceph-users] Re: Cluster down after network outage

On Wed, Jul 12, 2023 at 1:26 AM Frank Schilder 
mailto:fr...@dtu.dk>> wrote:

Hi all,

one problem solved, another coming up. For everyone ending up in the same 
situation, the trick seems to be to get all OSDs marked up and then allow 
recovery. Steps to take:

- set noout, nodown, norebalance, norecover
- wait patiently until all OSDs are shown as up
- unset norebalance, norecover
- wait wait wait, PGs will eventually become active as OSDs become responsive
- unset nodown, noout

Nice work

[ceph-users] resume RBD mirror on another host

[Tony Liu]

Hi,

How RBD mirror tracks mirroring process, on local storage?
Say, RBD mirror is running on host-1, when host-1 goes down,
start RBD mirror on host-2. In that case, is RBD mirror on host-2
going to continue the mirroring?


Thanks!
Tony
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Ceph Developer Summit - Squid

[Neha Ojha]

Hi everyone,

Join us for the Ceph Developer Summit happening virtually from Jul 17 – 24,
2023. We'll be discussing features planned for our next release, Squid,
across the course of this summit. The schedule has been published on our
website[0] and meetings have been added to the community calendar[1]. You
can find agenda items in this etherpad[2].

We look forward to seeing you there!

Thanks,
Neha

[0] https://ceph.io/en/community/events/2023/ceph-developer-summit-squid/
[1]
https://calendar.google.com/calendar/u/0/embed?src=9ts9c7lt7u1vic2ijvvqqlf...@group.calendar.google.com
[2] https://pad.ceph.com/p/cds-squid
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: resume RBD mirror on another host

[Ilya Dryomov]

On Thu, Jul 13, 2023 at 6:16 PM Tony Liu  wrote:
>
> Hi,
>
> How RBD mirror tracks mirroring process, on local storage?
> Say, RBD mirror is running on host-1, when host-1 goes down,
> start RBD mirror on host-2. In that case, is RBD mirror on host-2
> going to continue the mirroring?

Hi Tony,

No, it's tracked on the RBD image itself -- meaning in RADOS.

With journal-based mirroring it's the commit position in the journal
(roughly).

For snapshot-based mirroring, there is last_copied_object_number field
on each mirror snapshot.

Thanks,

Ilya
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: resume RBD mirror on another host

[Ilya Dryomov]

On Thu, Jul 13, 2023 at 10:23 PM Ilya Dryomov  wrote:
>
> On Thu, Jul 13, 2023 at 6:16 PM Tony Liu  wrote:
> >
> > Hi,
> >
> > How RBD mirror tracks mirroring process, on local storage?
> > Say, RBD mirror is running on host-1, when host-1 goes down,
> > start RBD mirror on host-2. In that case, is RBD mirror on host-2
> > going to continue the mirroring?
>
> Hi Tony,
>
> No, it's tracked on the RBD image itself -- meaning in RADOS.

To be clear, "no" here was meant to answer the "How RBD mirror
tracks mirroring process, on local storage?" question.

To answer the second question explicitly: yes, rbd-mirror daemon on
host-2 will continue mirroring from where rbd-mirror daemon on host-1
left off.

Thanks,

Ilya
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: resume RBD mirror on another host

[Tony Liu]

Super! Thanks Ilya!

Tony

From: Ilya Dryomov 
Sent: July 13, 2023 01:30 PM
To: Tony Liu
Cc: d...@ceph.io; ceph-users@ceph.io
Subject: Re: [ceph-users] resume RBD mirror on another host

On Thu, Jul 13, 2023 at 10:23 PM Ilya Dryomov  wrote:
>
> On Thu, Jul 13, 2023 at 6:16 PM Tony Liu  wrote:
> >
> > Hi,
> >
> > How RBD mirror tracks mirroring process, on local storage?
> > Say, RBD mirror is running on host-1, when host-1 goes down,
> > start RBD mirror on host-2. In that case, is RBD mirror on host-2
> > going to continue the mirroring?
>
> Hi Tony,
>
> No, it's tracked on the RBD image itself -- meaning in RADOS.

To be clear, "no" here was meant to answer the "How RBD mirror
tracks mirroring process, on local storage?" question.

To answer the second question explicitly: yes, rbd-mirror daemon on
host-2 will continue mirroring from where rbd-mirror daemon on host-1
left off.

Thanks,

Ilya
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Multisite sync - zone permission denied

[Szabo, Istvan (Agoda)]

Hi,

Have you had the issue with zones are permission denied?

failed to retrieve sync info: (13) Permission denied

It's a newly added zone, uses the same sync user and credentials but it shows 
permission denied and I don't see any reason behind.

Thank you


This message is confidential and is for the sole use of the intended 
recipient(s). It may also be privileged or otherwise protected by copyright or 
other legal rules. If you have received it by mistake please let us know by 
reply email and delete it from your system. It is prohibited to copy this 
message or disclose its content to anyone. Any confidentiality or privilege is 
not waived or lost by any mistaken delivery or unauthorized disclosure of the 
message. All messages sent to and from Agoda may be monitored to ensure 
compliance with company policies, to protect the company's interests and to 
remove potential malware. Electronic messages may be intercepted, amended, lost 
or deleted, or contain viruses.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io