[ceph-users] user and bucket not sync ( permission denied )

[Guillaume Morin]

Hello, i have an issue about my multisite configuration.

pacific 16.2.9


My problem:

i have a permission denied on the the master zone when i use the command below.


$ radosgw-admin sync status

realm 8df19226-a200-48fa-bd43-1491d32c636c (myrealm)

zonegroup 29592d75-224d-49b6-bc36-2703efa4f67f (myzonegroup)

zone 6cce41f3-a54b-47c2-981f-3b56ca0a4489 (myzone)

metadata sync no sync (zone is master)

2023-03-07T22:31:16.466+0100 7f96a3e7a840 0 ERROR: failed to fetch datalog info

data sync source: f2b20676-2672-4a92-a7ee-f3eb2efb12c6 (mysecondaryzone)

failed to retrieve sync info: (13) Permission denied


because on secondary zone (read only) , i see a 403 error about the permission 
denied from the master node

2023-03-07T00:00:53.309+0100 7f1ec8f21700 1 == starting new request 
req=0x7f1fd418c620 =

2023-03-07T00:00:53.309+0100 7f1ec8f21700 1 req 2604939314198041770 
0.0s op->ERRORHANDLER: err_no=-2028 new_err_no=-2028

2023-03-07T00:00:53.309+0100 7f1ec8f21700 1 == req done req=0x7f1fd418c620 
op status=0 http_status=403 latency=0.0s ==

2023-03-07T00:00:53.309+0100 7f1ec8f21700 1 beast: 0x7f1fd418c620: 10. 
- - [07/Mar/2023:00:00:53.309 +0100] "POST 
/admin/realm/period?period=395f9f13-d941-4ccf-a0cf-6c5d6d6579c2=76=29592d75-224d-49b6-bc36-2703efa4f67f
 HTTP/1.1" 403 194 - - - latency=0.0s

2023-03-07T00:00:53.441+0100 7f1e7e68c700 1 == starting new request 
req=0x7f1fd4411620 =

2023-03-07T00:00:53.441+0100 7f1e7e68c700 1 req 7374970752399537975 
0.0s op->ERRORHANDLER: err_no=-2028 new_err_no=-2028

2023-03-07T00:00:53.441+0100 7f1e7e68c700 1 == req done req=0x7f1fd4411620 
op status=0 http_status=403 latency=0.0s ==

2023-03-07T00:00:53.441+0100 7f1e7e68c700 1 beast: 0x7f1fd4411620: 10. 
- - [07/Mar/2023:00:00:53.441 +0100] "POST 
/admin/log?type=data=6cce41f3-a54b-47c2-981f-3b56ca0a4489=29592d75-224d-49b6-bc36-2703efa4f67f
 HTTP/1.1" 403 194 - - - latency=0.0s


No issue when i use the command to check sync on secondary zone


I don't understand because on secondary zone, pull realm and period with a user 
with flag system and admin works, the sync works for objects but not for users 
and buckets. When i list user and bucket on secondary zone, there are nothing 
but i have my objects on pool bucket.data !!



i think the 403 was due because my user with flag system doesn't exist on 
secondary zone but i don't understand why user and bucket are not syncronized 
??!!

Access key and secret key are set on master zone and secondary zone, endpoint 
also

I have an other cluster with a similary configuration and i don't have any issue

Can someone help me ?

Sorry for my english


Regards

Guillaume


___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Issue upgrading 17.2.0 to 17.2.5

[aellahib]

Your advice regarding the set container images manually did lead me to check 
cephadmin config to see what other nodes are set to and i did see stop and 
17.2.5 set for certain nodes and OSDs. As soon as I pointed all of them the 
right away my logs started showing real data and I can deploy and configure 
nodes.

Thank you very much for your help!
 I will attempt to upgrade again soon.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Issue upgrading 17.2.0 to 17.2.5

[aellahib]

Hey David, yes its me..Thank you for your help btw.
 I was waiting on my acceptance to the ceph tracker website. Seems it is in so 
I will submit a request soon, but I havent been able to reproduce it so I am 
not sure if I can provide relevant info for that.
I already ran that orch upgrade stop command multiple times, the new return i 
am getting ir not a stop image but rather 17.2.5 with some additional fields as 
I posted above, very strange.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Upgrade problem from 1.6 to 1.7

[Oğuz Yarımtepe]

Hi,

I have a cluster with rook operator running the ceph version 1.6 and
upgraded first rook operator and then the ceph cluster definition.
Everything was fine, every component except from osds are upgraded. Below
is the reason of OSD not being upgraded:

not updating OSD 1 on node "some-node-name". node no longer exists in the
storage spec. if the user wishes to remove OSDs from the node, they must do
so manually. Rook will not remove OSDs from nodes that are removed from the
storage spec in order to prevent accidental data loss

Any idea or anyone had seen it before?

Regards.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Issue upgrading 17.2.0 to 17.2.5

[Adam King]

>
> Current cluster status says healthy but I cannot deploy new daemons, the
>> mgr information isnt refreshing (5 days old info) under hosts and services
>> but the main dashboard is accurate like ceph -s
>> Ceph -s will show accurate information but things like ceph orch ps
>> --daemon-type mgr will say that I have 5MGRs running which is inaccurate,
>> nor will it let me remove them manually as it says theyre not found
>>
>
Can you try a mgr failover (ceph mgr fail), wait ~5 minutes and then see
what actually gets refreshed (as in check the refreshed column in "ceph
orch ps" and "ceph orch device ls"). Typically when it's having issues like
this where it's "stuck" and not refreshing there is an issue blocking the
refresh on one specific host, so would be good to see if most hosts refresh
and there is only specific host(s) where the refresh doesn't occur.

osd.11  basic
>  container_imagestop
>
osd.47  basic
>  container_image17.2.5
>*

osd.49  basic
>  container_image17.2.5
>*


That looks bad.  Might be worth trying just a "ceph config set osd
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346"
to get all the osd config options onto a valid image. With those options it
will try to use the image "stop" or "17.2.5" when redeploying or upgrading
those OSDs.

On Tue, Mar 7, 2023 at 11:40 AM  wrote:

> Hello at this point I've tried to upgrade a few times so I believe the
> command is long gone. On another forum someone was eluding that i
> accidentally set the image to "stop" instead of running a proper upgrade
> stop command but I couldnt find anything like that on the hosts I ran
> commands from but wouldnt be surprised if i accidentally pasted then wrote
> additional commands to it.
>
> The failing OSD was interesting, ceph didnt report it as a stray daemon
> but i noticed it was showing as a daemon but not as an actual OSD for
> storage in ceph, so I attempted to remove it and it would eventually come
> back.
>
> It had upgraded all the managers, mons to 17.2.5. Some OSDs had upgraded
> as well.
> Current cluster status says healthy but I cannot deploy new daemons, the
> mgr information isnt refreshing (5 days old info) under hosts and services
> but the main dashboard is accurate like ceph -s
> Ceph -s will show accurate information but things like ceph orch ps
> --daemon-type mgr will say that I have 5MGRs running which is inaccurate,
> nor will it let me remove them manually as it says theyre not found
>
> ERROR: Failed command: /usr/bin/docker pull 17.2.5
> 2023-03-06T09:26:55.925386-0700 mgr.mgr.idvkbw [DBG] serve loop sleep
> 2023-03-06T09:26:55.925507-0700 mgr.mgr.idvkbw [DBG] Sleeping for 60
> seconds
> 2023-03-06T09:27:55.925847-0700 mgr.mgr.idvkbw [DBG] serve loop wake
> 2023-03-06T09:27:55.925959-0700 mgr.mgr.idvkbw [DBG] serve loop start
> 2023-03-06T09:27:55.929849-0700 mgr.mgr.idvkbw [DBG] mon_command: 'config
> dump' -> 0 in 0.004s
> 2023-03-06T09:27:55.931625-0700 mgr.mgr.idvkbw [DBG] _run_cephadm :
> command = pull
> 2023-03-06T09:27:55.932025-0700 mgr.mgr.idvkbw [DBG] _run_cephadm : args =
> []
> 2023-03-06T09:27:55.932469-0700 mgr.mgr.idvkbw [DBG] args: --image 17.2.5
> --no-container-init pull
> 2023-03-06T09:27:55.932925-0700 mgr.mgr.idvkbw [DBG] Running command:
> which python3
> 2023-03-06T09:27:55.968793-0700 mgr.mgr.idvkbw [DBG] Running command:
> /usr/bin/python3
> /var/lib/ceph/5058e342-dac7-11ec-ada3-01065e90228d/cephadm.059bfc99f5cf36ed881f2494b104711faf4cbf5fc86a9594423cc105cafd9b4e
> --image 17.2.5 --no-container-init pull
> 2023-03-06T09:27:57.278932-0700 mgr.mgr.idvkbw [DBG] code: 1
> 2023-03-06T09:27:57.279045-0700 mgr.mgr.idvkbw [DBG] err: Pulling
> container image 17.2.5...
> Non-zero exit code 1 from /usr/bin/docker pull 17.2.5
> /usr/bin/docker: stdout Using default tag: latest
> /usr/bin/docker: stderr Error response from daemon: pull access denied for
> 17.2.5, repository does not exist or may require 'docker login': denied:
> requested access to the resource is denied
> ERROR: Failed command: /usr/bin/docker pull 17.2.5
>
> 2023-03-06T09:27:57.280517-0700 mgr.mgr.idvkbw [DBG] serve loop
>
> I had stopped the upgrade before so its at
> neteng@mon:~$ ceph orch upgrade status
> {
> "target_image": null,
> "in_progress": false,
> "which": "",
> "services_complete": [],
> "progress": null,
> "message": "",
> "is_paused": false
> }
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
___

[ceph-users] Re: upgrading from 15.2.17 to 16.2.11 - Health ERROR

[Adam King]

that looks like it was expecting a json structure somewhere and got a blank
string. Is there anything in the logs (ceph log last 100 info cephadm)? If
not, might be worth trying a couple mgr failovers (I'm assuming only one
got upgraded, so first failover would go back to the 15.2.17 one and then a
second failover would go back to the 16.2.11 one) and then rechecking the
logs. I'd expect this to generate a traceback there and it's hard to say
what happened without that.

On Tue, Mar 7, 2023 at 11:42 AM  wrote:

> hi , starting upgrade from 15.2.17 i got this error
> Module 'cephadm' has failed: Expecting value: line 1 column 1 (char 0)
>
> Cluster was in health ok before starting.
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] upgrading from 15.2.17 to 16.2.11 - Health ERROR

[xadhoom76]

hi , starting upgrade from 15.2.17 i got this error
Module 'cephadm' has failed: Expecting value: line 1 column 1 (char 0)

Cluster was in health ok before starting.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Issue upgrading 17.2.0 to 17.2.5

[aellahib]

This is the output

{
"target_image": null,
"in_progress": false,
"which": "",
"services_complete": [],
"progress": null,
"message": "",
"is_paused": false
}


grep image
global  basic 
container_image
quay.io/ceph/ceph@sha256:12a0a4f43413fd97a14a3d47a3451b2d2df50020835bb93db666209f3f77617a
  *
mon basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
mgr basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.0   basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.1   basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.11  basic 
container_imagestop 
  *
osd.16  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.17  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.2   basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.25  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.3   basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.34  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.35  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.37  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.38  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.39  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.40  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.42  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.43  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.44  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.45  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.46  basic 
container_image
quay.io/ceph/ceph@sha256:2b73ccc9816e0a1ee1dfbe21ba9a8cc085210f1220f597b5050ebfcac4bdd346
  *
osd.47  basic 
container_image17.2.5   
  *
osd.49  basic 
container_image 

[ceph-users] Re: Issue upgrading 17.2.0 to 17.2.5

[aellahib]

Hello at this point I've tried to upgrade a few times so I believe the command 
is long gone. On another forum someone was eluding that i accidentally set the 
image to "stop" instead of running a proper upgrade stop command but I couldnt 
find anything like that on the hosts I ran commands from but wouldnt be 
surprised if i accidentally pasted then wrote additional commands to it.

The failing OSD was interesting, ceph didnt report it as a stray daemon but i 
noticed it was showing as a daemon but not as an actual OSD for storage in 
ceph, so I attempted to remove it and it would eventually come back.

It had upgraded all the managers, mons to 17.2.5. Some OSDs had upgraded as 
well.
Current cluster status says healthy but I cannot deploy new daemons, the mgr 
information isnt refreshing (5 days old info) under hosts and services but the 
main dashboard is accurate like ceph -s
Ceph -s will show accurate information but things like ceph orch ps 
--daemon-type mgr will say that I have 5MGRs running which is inaccurate, nor 
will it let me remove them manually as it says theyre not found

ERROR: Failed command: /usr/bin/docker pull 17.2.5
2023-03-06T09:26:55.925386-0700 mgr.mgr.idvkbw [DBG] serve loop sleep
2023-03-06T09:26:55.925507-0700 mgr.mgr.idvkbw [DBG] Sleeping for 60 seconds
2023-03-06T09:27:55.925847-0700 mgr.mgr.idvkbw [DBG] serve loop wake
2023-03-06T09:27:55.925959-0700 mgr.mgr.idvkbw [DBG] serve loop start
2023-03-06T09:27:55.929849-0700 mgr.mgr.idvkbw [DBG] mon_command: 'config dump' 
-> 0 in 0.004s
2023-03-06T09:27:55.931625-0700 mgr.mgr.idvkbw [DBG] _run_cephadm : command = 
pull
2023-03-06T09:27:55.932025-0700 mgr.mgr.idvkbw [DBG] _run_cephadm : args = []
2023-03-06T09:27:55.932469-0700 mgr.mgr.idvkbw [DBG] args: --image 17.2.5 
--no-container-init pull
2023-03-06T09:27:55.932925-0700 mgr.mgr.idvkbw [DBG] Running command: which 
python3
2023-03-06T09:27:55.968793-0700 mgr.mgr.idvkbw [DBG] Running command: 
/usr/bin/python3 
/var/lib/ceph/5058e342-dac7-11ec-ada3-01065e90228d/cephadm.059bfc99f5cf36ed881f2494b104711faf4cbf5fc86a9594423cc105cafd9b4e
 --image 17.2.5 --no-container-init pull
2023-03-06T09:27:57.278932-0700 mgr.mgr.idvkbw [DBG] code: 1
2023-03-06T09:27:57.279045-0700 mgr.mgr.idvkbw [DBG] err: Pulling container 
image 17.2.5...
Non-zero exit code 1 from /usr/bin/docker pull 17.2.5
/usr/bin/docker: stdout Using default tag: latest
/usr/bin/docker: stderr Error response from daemon: pull access denied for 
17.2.5, repository does not exist or may require 'docker login': denied: 
requested access to the resource is denied
ERROR: Failed command: /usr/bin/docker pull 17.2.5

2023-03-06T09:27:57.280517-0700 mgr.mgr.idvkbw [DBG] serve loop

I had stopped the upgrade before so its at
neteng@mon:~$ ceph orch upgrade status
{
"target_image": null,
"in_progress": false,
"which": "",
"services_complete": [],
"progress": null,
"message": "",
"is_paused": false
}
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Theory about min_size and its implications

[stefan . pinter]

hi!

thanks to all of you, I appreciate this very much! I will have to go through 
all of your messages a few more times and do some research.

so our rule from the intial post does make sure, that, when 1 room goes down it 
does NOT try to restore 3 replicas in the remaining room but it will only try 
to create a second replica for each PG that only has 1 at the moment of the 
outage?
i mean: will the ceph cluster try to restore 2 replicas of each PG in the 
remaining room when every OSD from room 1 is marked as out?
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] upgrade problem from 1.6 to 1.7 related with osd

[Oğuz Yarımtepe]

Hi,

I have a cluster with rook operator running the ceph version 1.6 and
upgraded first rook operator and then the ceph cluster definition.
Everything was fine, every component except from osds are upgraded. Below
is the reason of OSD not being upgraded:

not updating OSD 1 on node "some-node-name". node no longer exists in the
storage spec. if the user wishes to remove OSDs from the node, they must do
so manually. Rook will not remove OSDs from nodes that are removed from the
storage spec in order to prevent accidental data loss

Any idea or anyone had seen it before?

Regards.

-- 
Oğuz Yarımtepe
http://about.me/oguzy
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Problem with cephadm and deploying 4 ODSs on nvme Storage

[claas . goltz]

This post took a while to be checked from a moderator and meanwhile I found a 
Service rule, that fetched all my available diskes. I deleted it and after 
that, all commands works as foreseen.

Thanks to all for reading.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: mds readonly, mds all down

[kreept . sama]

Sorry, can anyone aks how to fix MountVolume.MountDevice failed for volume 
"pvc-7b60b096-c9d3-4b6f-a8fc-5241541959d3" : rpc error: code = Internal desc = 
rados: ret=-61, No data available: "error in getxattr"
Now its a main problem 
Thank you
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Role for setting quota on Cephfs pools

[saaa_2001]

Hello, 

I need to provide a credential to set of users to allow them to set quota for 
Cephfs pools. How do I accomplish it?
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] restoring ceph cluster from osds

[Ben]

Hi,

I ended up with having whole set of osds to get back original ceph cluster.
I figured out to make the cluster running. However, it's status is
something as below:

bash-4.4$ ceph -s

  cluster:

id: 3f271841-6188-47c1-b3fd-90fd4f978c76

health: HEALTH_WARN

7 daemons have recently crashed

4 slow ops, oldest one blocked for 35077 sec, daemons
[mon.a,mon.b] have slow ops.



  services:

mon: 3 daemons, quorum a,b,d (age 9h)

mgr: b(active, since 14h), standbys: a

osd: 4 osds: 0 up, 4 in (since 9h)



  data:

pools:   0 pools, 0 pgs

objects: 0 objects, 0 B

usage:   0 B used, 0 B / 0 B avail

pgs:


All osds are down.


I checked the osds logs and attached with this.


Please help and I wonder if it's possible to get the cluster back. I have
some backup for monitor's data. Till now I haven't restore that in the
course.


Thanks,

Ben
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Creating a role for quota management

[Adiga, Anantha]

Thank you Xiubo, will try that option, looks like it is done with the intention 
to keep it at the client level.

Anantha

-Original Message-
From: Xiubo Li  
Sent: Tuesday, March 7, 2023 12:44 PM
To: Adiga, Anantha ; ceph-users@ceph.io
Subject: Re: [ceph-users] Creating a role for quota management

Hi

Maybe you can use the rule of CEPHFS CLIENT CAPABILITIES and only enable the 
'p' permission for some users, which will allow them to SET_VXATTR.

I didn't find the similar cap from the OSD CAPBILITIES.

Thanks

On 07/03/2023 00:33, anantha.ad...@intel.com wrote:
> Hello,
>
> Can you provide details on how to create a role for allowing a set of users 
> to set quota on CephFS pools ?
> ___
> ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an 
> email to ceph-users-le...@ceph.io
>

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Very slow backfilling

[Sridhar Seshasayee]

Hi Joffrey,

That's good to know. Please note that you can switch back to
"high_client_ops" profile
once all the recoveries are completed. This is to ensure clients operations
get
higher priority when there aren't many recoveries or no recoveries going on.

-Sridhar

On Tue, Mar 7, 2023 at 2:51 PM Joffrey  wrote:

> Hi, I changed de mclock priority last Friday with "ceph tell 'osd.*'
> injectargs '--osd_mclock_profile=high_recovery_ops'" and now, the Health is
> OK.
> So, you(re right, I need to change the mclock scheduler to changes
> recovery priority.
>
> Thanks you
>
> Le sam. 4 mars 2023 à 08:12, Sridhar Seshasayee  a
> écrit :
>
>> Hello  Joffrey,
>> You could be hitting the slow backfill/recovery issue with
>> mclock_scheduler.
>> To confirm the above could you please provide the output of the following
>> commands?
>>
>> 1. ceph versions
>> 2. ceph config show osd. | grep osd_max_backfills
>> 3. ceph config show osd. | grep osd_recovery_max_active
>> 4. ceph config show osd. | grep osd_mclock
>>
>> where 'id' can be any valid osd id.
>>
>> With the mclock_scheduler enabled and with 17.2.5, it is not possible to
>> override recovery settings like 'osd_max_backfills' and other recovery
>> related config options.
>>
>> -Sridhar
>> ___
>> ceph-users mailing list -- ceph-users@ceph.io
>> To unsubscribe send an email to ceph-users-le...@ceph.io
>>
>

-- 

Sridhar Seshasayee

Partner Engineer

Red Hat 

___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Very slow backfilling

[Joffrey]

Hi, I changed de mclock priority last Friday with "ceph tell 'osd.*'
injectargs '--osd_mclock_profile=high_recovery_ops'" and now, the Health is
OK.
So, you(re right, I need to change the mclock scheduler to changes recovery
priority.

Thanks you

Le sam. 4 mars 2023 à 08:12, Sridhar Seshasayee  a
écrit :

> Hello  Joffrey,
> You could be hitting the slow backfill/recovery issue with
> mclock_scheduler.
> To confirm the above could you please provide the output of the following
> commands?
>
> 1. ceph versions
> 2. ceph config show osd. | grep osd_max_backfills
> 3. ceph config show osd. | grep osd_recovery_max_active
> 4. ceph config show osd. | grep osd_mclock
>
> where 'id' can be any valid osd id.
>
> With the mclock_scheduler enabled and with 17.2.5, it is not possible to
> override recovery settings like 'osd_max_backfills' and other recovery
> related config options.
>
> -Sridhar
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io