Are you running tight on RAM?
You might be running into http://tracker.ceph.com/issues/22464
Paul
2018-06-28 17:17 GMT+02:00 Bryan Banister :
> Hi all,
>
>
>
> We started running a EC pool based object store, set up with a 4+2
> configuration, and we seem to be getting an almost constant report of
> inconsistent PGs during scrub operations. For example:
>
> root@rook-tools:/# ceph pg ls inconsistent
>
> PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES
> LOG DISK_LOG STATE STATE_STAMP
> VERSION REPORTED UP UP_PRIMARY
> ACTING ACTING_PRIMARY LAST_SCRUB
> SCRUB_STAMPLAST_DEEP_SCRUB DEEP_SCRUB_STAMP
>
> 19.26 104 00 0 0 436207616
> 1537 1537 active+clean+inconsistent 2018-06-28 15:04:54.054227
> 1811'3137 2079:5075 [206,116,68,31,193,156]206
> [206,116,68,31,193,156]206 1811'3137 2018-06-27
> 21:00:17.011611 1811'3137 2018-06-27 21:00:17.011611
>
> 19.234 98 00 0 0 406847488
> 1581 1581 active+clean+inconsistent 2018-06-28 15:05:18.077003
> 1811'2981 2080:4822 [28,131,229,180,84,68] 28
> [28,131,229,180,84,68] 28 1811'2981 2018-06-28
> 14:09:54.092401 1811'2981 2018-06-28 14:09:54.092401
>
> 19.2a8 116 00 0 0 486539264
> 1561 1561 active+clean+inconsistent 2018-06-28 15:04:54.073762
> 1811'3161 2079:4825 [177,68,222,13,131,107]177
> [177,68,222,13,131,107]177 1811'3161 2018-06-28
> 07:51:21.109587 1811'3161 2018-06-28 07:51:21.109587
>
> 19.406 126 00 0 0 520233399
> 1557 1557 active+clean+inconsistent 2018-06-28 15:04:57.142651
> 1811'3057 2080:4944 [230,199,128,68,92,11]230
> [230,199,128,68,92,11]230 1811'3057 2018-06-27
> 18:36:18.497899 1811'3057 2018-06-27 18:36:18.497899
>
> 19.46b 109 00 0 0 449840274
> 1558 1558 active+clean+inconsistent 2018-06-28 15:04:54.227970
> 1811'3058 2079:4986 [18,68,130,94,181,225] 18
> [18,68,130,94,181,225] 18 1811'3058 2018-06-27
> 14:32:17.800961 1811'3058 2018-06-27 14:32:17.800961
>
> [snip]
>
>
>
> We sometimes see that running a deep scrub on the PG resolves the issue
> but not all the time.
>
>
>
> We have been running the PG repair operation on them (e.g. ceph pg repair
> ), which clears the issue. Is this the correct way to resolve this
> issue?
>
>
>
> Is this a normal behavior for a Ceph cluster?
>
>
>
> If so, why doesn’t it attempt to repair itself automatically?
>
>
>
> Thanks for the help understanding Ceph, we are very new to it still!
>
> -Bryan
>
>
>
>
>
> --
>
> Note: This email is for the confidential use of the named addressee(s)
> only and may contain proprietary, confidential, or privileged information
> and/or personal data. If you are not the intended recipient, you are hereby
> notified that any review, dissemination, or copying of this email is
> strictly prohibited, and requested to notify the sender immediately and
> destroy this email and any attachments. Email transmission cannot be
> guaranteed to be secure or error-free. The Company, therefore, does not
> make any guarantees as to the completeness or accuracy of this email or any
> attachments. This email is for informational purposes only and does not
> constitute a recommendation, offer, request, or solicitation of any kind to
> buy, sell, subscribe, redeem, or perform any type of transaction of a
> financial product. Personal data, as defined by applicable data privacy
> laws, contained in this email may be processed by the Company, and any of
> its affiliated or related companies, for potential ongoing compliance
> and/or business-related purposes. You may have rights regarding your
> personal data; for information on exercising these rights or the Company’s
> treatment of personal data, please email datareque...@jumptrading.com.
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
--
Paul Emmerich
Looking for help with your Ceph cluster? Contact us at https://croit.io
croit GmbH
Freseniusstr. 31h
81247 München
www.croit.io
Tel: +49 89 1896585 90
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
