Dear all,
I have a ceph cluster running in 3 nodes, 240 TB space with 60% usage,
used by rbd and radosgw clients. Recently I upgraded from emperor to
firefly, and I got the message about legacy tunables described in
http://ceph.com/docs/master/rados/operations/crush-map/#tunables. After
some data rearrangement to minimize risks, I tried to apply the optimal
settings. This resulted in 28% of object degradation, much more than I
expected, and worse, I lost communication for the rbd clients, running
in kernels 3.10 or 3.11.
Searching for a solution, I got to this proposed solution:
https://www.mail-archive.com/ceph-users@lists.ceph.com/msg11199.html.
Applying it (before the data was all moved), I got additional 2% of
object degradation, but the rbd clients came back into working. But then
I got a large number of degraded or staled PGs, that are not
backfilling. Looking for the definition of chooseleaf_vary_r, I reached
the definition in http://ceph.com/docs/master/rados/operations/crush-map/:
"chooseleaf_vary_r: Whether a recursive chooseleaf attempt will start
with a non-zero value of r, based on how many attempts the parent has
already made. Legacy default is 0, but with this value CRUSH is
sometimes unable to find a mapping. The optimal value (in terms of
computational cost and correctness) is 1. However, for legacy clusters
that have lots of existing data, changing from 0 to 1 will cause a lot
of data to move; a value of 4 or 5 will allow CRUSH to find a valid
mapping but will make less data move."
Is there any suggestion to handle it? Have I to set chooseleaf_vary_r to
some other value? Will I lose communication with my rbd clients? Or
should I return to legacy tunables?
Regards,
Gerd Jakobovitsch
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
