[ceph-users] obj_size_info_mismatch error handling

2019-04-29 Thread Reed Dier
Hi list, Woke up this morning to two PG's reporting scrub errors, in a way that I haven't seen before. > $ ceph versions > { > "mon": { > "ceph version 13.2.5 (cbff874f9007f1869bfd3821b7e33b2a6ffd4988) mimic > (stable)": 3 > }, > "mgr": { > "ceph version 13.2.5 (cbff8

Re: [ceph-users] obj_size_info_mismatch error handling

2019-04-30 Thread ceph
Hello Reed, I would give PG repair a try. IIRC there should be issue when you have Size 3... it would be difficult when you have Size 2 I guess... Hth Mehmet Am 29. April 2019 17:05:48 MESZ schrieb Reed Dier : >Hi list, > >Woke up this morning to two PG's reporting scrub errors, in a way that >

Re: [ceph-users] obj_size_info_mismatch error handling

2019-04-30 Thread Brad Hubbard
Which size is correct? On Tue, Apr 30, 2019 at 1:06 AM Reed Dier wrote: > > Hi list, > > Woke up this morning to two PG's reporting scrub errors, in a way that I > haven't seen before. > > $ ceph versions > { > "mon": { > "ceph version 13.2.5 (cbff874f9007f1869bfd3821b7e33b2a6ffd4988

Re: [ceph-users] obj_size_info_mismatch error handling

2019-04-30 Thread Brad Hubbard
On Wed, May 1, 2019 at 10:54 AM Brad Hubbard wrote: > > Which size is correct? Sorry, accidental discharge =D If the object info size is *incorrect* try forcing a write to the OI with something like the following. 1. rados -p [name_of_pool_17] setomapval 10008536718. temporary-key anyth

Re: [ceph-users] obj_size_info_mismatch error handling

2019-05-03 Thread Reed Dier
Just to follow up for the sake of the mailing list, I had not had a chance to attempt your steps yet, but things appear to have worked themselves out on their own. Both scrub errors cleared without intervention, and I'm not sure if it is the results of that object getting touched in CephFS that

Re: [ceph-users] obj_size_info_mismatch error handling

2019-06-03 Thread Dan van der Ster
Hi Reed and Brad, Did you ever learn more about this problem? We currently have a few inconsistencies arriving with the same env (cephfs, v13.2.5) and symptoms. PG Repair doesn't fix the inconsistency, nor does Brad's omap workaround earlier in the thread. In our case, we can fix by cp'ing the fi

Re: [ceph-users] obj_size_info_mismatch error handling

2019-06-06 Thread Reed Dier
Sadly I never discovered anything more. It ended up clearing up on its own, which was disconcerting, but I resigned to not making things worse in an attempt to make them better. I assume someone touched the file in CephFS, which triggered the metadata to be updated, and everyone was able to rea

Re: [ceph-users] obj_size_info_mismatch error handling

2019-06-17 Thread Brad Hubbard
Can you open a tracker for this Dan and provide scrub logs with debug_osd=20 and rados list-inconsistent-obj output? On Mon, Jun 3, 2019 at 10:44 PM Dan van der Ster wrote: > > Hi Reed and Brad, > > Did you ever learn more about this problem? > We currently have a few inconsistencies arriving wit