[ceph-users] Re: inconsistent pg after upgrade nautilus to octopus

2021-12-19 Thread Christian Rohmann
Hello Tomasz, I observe a strange accumulation of inconsistencies for an RGW-only (+multisite) setup, with errors just like those you reported. I collected some info and raised a bug ticket:  https://tracker.ceph.com/issues/53663 Two more inconsistencies have just shown up hours after

[ceph-users] Re: inconsistent pg after upgrade nautilus to octopus

2021-10-20 Thread Szabo, Istvan (Agoda)
Have you tried to repair pg? Istvan Szabo Senior Infrastructure Engineer --- Agoda Services Co., Ltd. e: istvan.sz...@agoda.com --- On 2021. Oct 20., at 9:04, Glaza

[ceph-users] Re: inconsistent pg after upgrade nautilus to octopus

2021-10-20 Thread Tomasz Płaza
Sorry Marc, didn't see second question. As the upgrade process states, rgw are the last one to be upgraded, so they are still on nautilus (centos7). Those logs showed up after upgrade of the first osd host. It is a multisite setup so I am a little afraid of upgrading rgw now. Etienne:

[ceph-users] Re: inconsistent pg after upgrade nautilus to octopus

2021-10-20 Thread Tomasz Płaza
I did it only on MON servers. OSDs are on centos 7. Process was: 1. stop mon 2. backup /var/lib/ceph 3. reinstall server as centos 8 and install ceph nautilus 4. restore /var/lib/ceph  and start mo 5. wait few days 6. upgrade mon to octopus On 20.10.2021 o 09:51, Marc wrote: How did you do the

[ceph-users] Re: inconsistent pg after upgrade nautilus to octopus

2021-10-20 Thread Tomasz Płaza
Yes I did, and despite "Too many repaired reads on 1 OSDs" health is back to HEALTH_OK. But it is second time it happened and do not know, should I go forward with update or hold on. Or maybe it is a bad move makeing compaction right after migration to 15.2.14 On 20.10.2021 o 09:21, Szabo,

[ceph-users] Re: inconsistent pg after upgrade nautilus to octopus

2021-10-20 Thread Marc
How did you do the upgrade from centos7 to centos8? I assume you kept osd config's etc? > upgrading nautilus (14.2.22) to octopus (15.2.14) on centos7 (Mon/Mgr > were additionally migrated to centos8 beforehand). Each day I upgraded > one host and after all osds were up, I manually compacted

[ceph-users] Re: inconsistent pg after upgrade nautilus to octopus

2021-10-20 Thread Etienne Menguy
Hi, You should check for inconsistency root cause. https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-pg/#pgs-inconsistent - Etienne Menguy etienne.men...@croit.io > On 20 Oct