My full OSD list (also here as pastebin https://paste.ubuntu.com/p/XJ4Pjm92B5/ )
ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS 14 hdd 9.09470 1.00000 9.1 TiB 6.9 TiB 6.8 TiB 71 KiB 18 GiB 2.2 TiB 75.34 1.04 69 up 19 hdd 9.09470 1.00000 9.1 TiB 6.9 TiB 6.8 TiB 80 KiB 18 GiB 2.2 TiB 75.33 1.04 72 up 22 hdd 9.09470 1.00000 9.1 TiB 8.0 TiB 8.0 TiB 80 KiB 21 GiB 1.1 TiB 88.13 1.22 84 up 25 hdd 9.09470 1.00000 9.1 TiB 7.9 TiB 7.9 TiB 3.7 MiB 21 GiB 1.2 TiB 87.06 1.20 85 up 30 hdd 9.09470 1.00000 9.1 TiB 6.9 TiB 6.9 TiB 48 KiB 18 GiB 2.2 TiB 76.21 1.05 73 up 33 hdd 9.09470 1.00000 9.1 TiB 7.2 TiB 7.2 TiB 56 KiB 19 GiB 1.9 TiB 79.13 1.09 76 up 34 hdd 9.09470 1.00000 9.1 TiB 6.7 TiB 6.6 TiB 84 KiB 17 GiB 2.4 TiB 73.13 1.01 70 up 35 hdd 9.09470 1.00000 9.1 TiB 7.2 TiB 7.2 TiB 120 KiB 19 GiB 1.9 TiB 79.63 1.10 74 up 12 hdd 9.09470 1.00000 9.1 TiB 7.6 TiB 7.6 TiB 136 KiB 20 GiB 1.4 TiB 84.10 1.16 76 up 16 hdd 9.09470 1.00000 9.1 TiB 7.1 TiB 7.1 TiB 92 KiB 18 GiB 2.0 TiB 77.86 1.07 76 up 17 hdd 9.09470 1.00000 9.1 TiB 7.3 TiB 7.3 TiB 52 KiB 19 GiB 1.8 TiB 80.48 1.11 73 up 20 hdd 9.09470 1.00000 9.1 TiB 6.9 TiB 6.9 TiB 84 KiB 18 GiB 2.2 TiB 75.58 1.04 70 up 23 hdd 9.09470 1.00000 9.1 TiB 5.8 TiB 5.8 TiB 104 KiB 15 GiB 3.3 TiB 64.02 0.88 63 up 26 hdd 9.09470 1.00000 9.1 TiB 7.4 TiB 7.4 TiB 16 KiB 20 GiB 1.7 TiB 81.31 1.12 79 up 28 hdd 9.09470 1.00000 9.1 TiB 6.1 TiB 6.1 TiB 48 KiB 17 GiB 3.0 TiB 67.02 0.92 67 up 31 hdd 9.09470 1.00000 9.1 TiB 5.9 TiB 5.9 TiB 84 KiB 16 GiB 3.2 TiB 64.69 0.89 62 up 13 hdd 9.09470 1.00000 9.1 TiB 7.8 TiB 7.8 TiB 64 KiB 20 GiB 1.3 TiB 86.17 1.19 80 up 15 hdd 9.09470 1.00000 9.1 TiB 6.7 TiB 6.7 TiB 36 KiB 18 GiB 2.4 TiB 73.89 1.02 73 up 18 hdd 9.09470 1.00000 9.1 TiB 7.5 TiB 7.5 TiB 72 KiB 20 GiB 1.6 TiB 82.47 1.14 80 up 21 hdd 9.09470 1.00000 9.1 TiB 7.9 TiB 7.9 TiB 44 KiB 21 GiB 1.2 TiB 87.23 1.20 83 up 24 hdd 9.09470 1.00000 9.1 TiB 6.9 TiB 6.9 TiB 104 KiB 18 GiB 2.2 TiB 76.07 1.05 71 up 27 hdd 9.09470 1.00000 9.1 TiB 6.7 TiB 6.7 TiB 56 KiB 17 GiB 2.4 TiB 74.08 1.02 72 up 29 hdd 9.09470 1.00000 9.1 TiB 6.3 TiB 6.3 TiB 56 KiB 17 GiB 2.8 TiB 69.46 0.96 68 up 32 hdd 9.09470 1.00000 9.1 TiB 8.0 TiB 8.0 TiB 112 KiB 21 GiB 1.1 TiB 88.02 1.21 84 up 37 hdd 9.09470 1.00000 9.1 TiB 7.4 TiB 7.4 TiB 76 KiB 19 GiB 1.7 TiB 81.69 1.13 77 up 39 hdd 9.09470 1.00000 9.1 TiB 7.4 TiB 7.4 TiB 32 KiB 20 GiB 1.6 TiB 81.90 1.13 76 up 41 hdd 9.09470 1.00000 9.1 TiB 6.8 TiB 6.8 TiB 93 KiB 18 GiB 2.3 TiB 74.41 1.03 73 up 43 hdd 9.09470 1.00000 9.1 TiB 6.4 TiB 6.4 TiB 72 KiB 17 GiB 2.7 TiB 70.42 0.97 66 up 45 hdd 9.09470 1.00000 9.1 TiB 6.5 TiB 6.5 TiB 48 KiB 17 GiB 2.6 TiB 71.56 0.99 67 up 46 hdd 9.09470 1.00000 9.1 TiB 6.9 TiB 6.9 TiB 104 KiB 18 GiB 2.2 TiB 76.08 1.05 71 up 48 hdd 9.09470 1.00000 9.1 TiB 7.3 TiB 7.3 TiB 40 KiB 19 GiB 1.8 TiB 80.13 1.11 78 up 50 hdd 9.09470 1.00000 9.1 TiB 6.4 TiB 6.4 TiB 76 KiB 17 GiB 2.7 TiB 70.39 0.97 68 up 36 hdd 9.09470 1.00000 9.1 TiB 6.8 TiB 6.8 TiB 92 KiB 18 GiB 2.3 TiB 74.88 1.03 72 up 38 hdd 9.09470 1.00000 9.1 TiB 7.0 TiB 6.9 TiB 80 KiB 18 GiB 2.1 TiB 76.55 1.06 75 up 40 hdd 9.09470 1.00000 9.1 TiB 7.5 TiB 7.5 TiB 96 KiB 19 GiB 1.6 TiB 82.46 1.14 76 up 42 hdd 9.09470 1.00000 9.1 TiB 7.1 TiB 7.1 TiB 80 KiB 19 GiB 2.0 TiB 78.51 1.08 78 up 44 hdd 9.09470 1.00000 9.1 TiB 8.0 TiB 8.0 TiB 104 KiB 21 GiB 1.1 TiB 88.01 1.21 85 up 47 hdd 9.09470 1.00000 9.1 TiB 6.3 TiB 6.3 TiB 112 KiB 17 GiB 2.8 TiB 69.59 0.96 67 up 49 hdd 9.09470 1.00000 9.1 TiB 6.1 TiB 6.0 TiB 12 KiB 16 GiB 3.0 TiB 66.69 0.92 66 up 51 hdd 9.09470 1.00000 9.1 TiB 6.0 TiB 6.0 TiB 76 KiB 16 GiB 3.1 TiB 66.16 0.91 64 up 52 hdd 9.09470 1.00000 9.1 TiB 8.0 TiB 7.9 TiB 3.7 MiB 20 GiB 1.1 TiB 87.63 1.21 83 up 53 hdd 9.09470 1.00000 9.1 TiB 8.1 TiB 8.1 TiB 100 KiB 21 GiB 1.0 TiB 88.77 1.22 85 up 54 hdd 9.09470 1.00000 9.1 TiB 7.1 TiB 7.1 TiB 64 KiB 19 GiB 2.0 TiB 78.51 1.08 76 up 55 hdd 9.09470 1.00000 9.1 TiB 7.4 TiB 7.3 TiB 60 KiB 19 GiB 1.7 TiB 80.90 1.12 74 up 56 hdd 9.09470 1.00000 9.1 TiB 7.9 TiB 7.9 TiB 48 KiB 21 GiB 1.2 TiB 87.09 1.20 82 up 57 hdd 9.09470 1.00000 9.1 TiB 7.0 TiB 7.0 TiB 48 KiB 19 GiB 2.1 TiB 76.82 1.06 72 up 58 hdd 9.09470 1.00000 9.1 TiB 8.0 TiB 8.0 TiB 56 KiB 21 GiB 1.1 TiB 88.11 1.22 83 up 59 hdd 9.09470 1.00000 9.1 TiB 7.9 TiB 7.8 TiB 72 KiB 20 GiB 1.2 TiB 86.34 1.19 85 up 60 hdd 9.09470 1.00000 9.1 TiB 6.9 TiB 6.9 TiB 72 KiB 18 GiB 2.2 TiB 75.94 1.05 72 up 61 hdd 9.09470 1.00000 9.1 TiB 7.4 TiB 7.4 TiB 144 KiB 19 GiB 1.6 TiB 81.87 1.13 79 up 62 hdd 9.09470 1.00000 9.1 TiB 6.3 TiB 6.3 TiB 72 KiB 16 GiB 2.8 TiB 69.36 0.96 64 up 63 hdd 9.09470 1.00000 9.1 TiB 8.2 TiB 8.1 TiB 72 KiB 21 GiB 958 GiB 89.71 1.24 88 up 64 hdd 9.09470 1.00000 9.1 TiB 6.5 TiB 6.5 TiB 56 KiB 18 GiB 2.6 TiB 71.27 0.98 66 up 65 hdd 9.09470 1.00000 9.1 TiB 7.8 TiB 7.8 TiB 68 KiB 21 GiB 1.3 TiB 86.01 1.19 81 up 66 hdd 9.09470 1.00000 9.1 TiB 7.5 TiB 7.4 TiB 100 KiB 19 GiB 1.6 TiB 81.97 1.13 80 up 67 hdd 9.09470 1.00000 9.1 TiB 7.9 TiB 7.9 TiB 72 KiB 21 GiB 1.2 TiB 87.20 1.20 84 up 84 hdd 9.09470 1.00000 9.1 TiB 7.8 TiB 7.8 TiB 136 KiB 20 GiB 1.3 TiB 86.14 1.19 81 up 85 hdd 9.09470 1.00000 9.1 TiB 8.2 TiB 8.1 TiB 24 KiB 21 GiB 958 GiB 89.71 1.24 89 up 86 hdd 9.09470 1.00000 9.1 TiB 6.8 TiB 6.8 TiB 56 KiB 18 GiB 2.3 TiB 75.06 1.04 75 up 87 hdd 9.09470 1.00000 9.1 TiB 7.9 TiB 7.9 TiB 100 KiB 20 GiB 1.2 TiB 87.18 1.20 82 up 88 hdd 9.09470 1.00000 9.1 TiB 7.1 TiB 7.1 TiB 124 KiB 18 GiB 2.0 TiB 78.31 1.08 76 up 89 hdd 9.09470 1.00000 9.1 TiB 8.1 TiB 8.1 TiB 72 KiB 21 GiB 1.0 TiB 88.95 1.23 87 up 90 hdd 9.09470 1.00000 9.1 TiB 6.7 TiB 6.7 TiB 88 KiB 18 GiB 2.4 TiB 74.15 1.02 70 up 91 hdd 9.09470 1.00000 9.1 TiB 6.7 TiB 6.7 TiB 32 KiB 18 GiB 2.4 TiB 73.82 1.02 70 up 76 hdd 9.09470 1.00000 9.1 TiB 6.8 TiB 6.8 TiB 60 KiB 18 GiB 2.3 TiB 75.25 1.04 73 up 77 hdd 9.09470 1.00000 9.1 TiB 7.6 TiB 7.6 TiB 100 KiB 20 GiB 1.5 TiB 83.94 1.16 81 up 78 hdd 9.09470 1.00000 9.1 TiB 6.9 TiB 6.9 TiB 56 KiB 18 GiB 2.2 TiB 76.04 1.05 74 up 79 hdd 9.09470 1.00000 9.1 TiB 6.6 TiB 6.6 TiB 68 KiB 17 GiB 2.5 TiB 72.83 1.00 69 up 80 hdd 9.09470 1.00000 9.1 TiB 6.7 TiB 6.7 TiB 100 KiB 18 GiB 2.4 TiB 73.85 1.02 70 up 81 hdd 9.09470 1.00000 9.1 TiB 6.7 TiB 6.7 TiB 76 KiB 18 GiB 2.4 TiB 73.83 1.02 69 up 82 hdd 9.09470 1.00000 9.1 TiB 7.7 TiB 7.6 TiB 108 KiB 20 GiB 1.4 TiB 84.28 1.16 77 up 83 hdd 9.09470 1.00000 9.1 TiB 8.1 TiB 8.0 TiB 112 KiB 21 GiB 1.0 TiB 88.55 1.22 86 up 68 hdd 9.09470 1.00000 9.1 TiB 7.5 TiB 7.5 TiB 40 KiB 20 GiB 1.6 TiB 82.27 1.13 78 up 69 hdd 9.09470 1.00000 9.1 TiB 7.2 TiB 7.2 TiB 120 KiB 19 GiB 1.9 TiB 79.04 1.09 78 up 70 hdd 9.09470 1.00000 9.1 TiB 7.1 TiB 7.1 TiB 44 KiB 19 GiB 2.0 TiB 77.80 1.07 73 up 71 hdd 9.09470 1.00000 9.1 TiB 6.0 TiB 6.0 TiB 32 KiB 16 GiB 3.1 TiB 66.41 0.92 57 up 72 hdd 9.09470 1.00000 9.1 TiB 5.7 TiB 5.7 TiB 84 KiB 15 GiB 3.3 TiB 63.20 0.87 59 up 73 hdd 9.09470 1.00000 9.1 TiB 5.9 TiB 5.9 TiB 60 KiB 16 GiB 3.2 TiB 65.04 0.90 63 up 74 hdd 9.09470 1.00000 9.1 TiB 6.0 TiB 5.9 TiB 56 KiB 16 GiB 3.1 TiB 65.55 0.90 66 up 75 hdd 9.09470 1.00000 9.1 TiB 5.5 TiB 5.5 TiB 84 KiB 14 GiB 3.6 TiB 60.53 0.83 53 up 121 hdd 9.09470 1.00000 9.1 TiB 6.2 TiB 6.1 TiB 96 KiB 17 GiB 2.9 TiB 67.70 0.93 62 up 123 hdd 9.09470 1.00000 9.1 TiB 7.1 TiB 7.1 TiB 44 KiB 19 GiB 2.0 TiB 77.89 1.07 73 up 124 hdd 9.09470 1.00000 9.1 TiB 5.3 TiB 5.3 TiB 60 KiB 15 GiB 3.8 TiB 57.94 0.80 56 up 126 hdd 9.09470 1.00000 9.1 TiB 7.5 TiB 7.4 TiB 68 KiB 19 GiB 1.6 TiB 82.10 1.13 77 up 128 hdd 9.09470 1.00000 9.1 TiB 6.3 TiB 6.3 TiB 84 KiB 17 GiB 2.8 TiB 69.45 0.96 69 up 130 hdd 9.09470 1.00000 9.1 TiB 6.2 TiB 6.1 TiB 104 KiB 16 GiB 2.9 TiB 67.72 0.93 66 up 132 hdd 9.09470 1.00000 9.1 TiB 7.2 TiB 7.2 TiB 60 KiB 19 GiB 1.9 TiB 79.20 1.09 79 up 134 hdd 9.09470 1.00000 9.1 TiB 6.6 TiB 6.6 TiB 92 KiB 17 GiB 2.5 TiB 72.92 1.01 70 up 112 hdd 9.09470 1.00000 9.1 TiB 6.9 TiB 6.9 TiB 88 KiB 18 GiB 2.2 TiB 75.95 1.05 74 up 113 hdd 9.09470 1.00000 9.1 TiB 6.1 TiB 6.0 TiB 104 KiB 16 GiB 3.0 TiB 66.58 0.92 67 up 114 hdd 9.09470 1.00000 9.1 TiB 5.9 TiB 5.9 TiB 116 KiB 16 GiB 3.2 TiB 64.90 0.90 59 up 115 hdd 9.09470 1.00000 9.1 TiB 7.8 TiB 7.8 TiB 128 KiB 21 GiB 1.3 TiB 85.98 1.19 81 up 116 hdd 9.09470 1.00000 9.1 TiB 6.5 TiB 6.4 TiB 52 KiB 17 GiB 2.6 TiB 70.94 0.98 70 up 117 hdd 9.09470 1.00000 9.1 TiB 7.8 TiB 7.7 TiB 32 KiB 20 GiB 1.3 TiB 85.37 1.18 83 up 118 hdd 9.09470 1.00000 9.1 TiB 7.1 TiB 7.0 TiB 48 KiB 19 GiB 2.0 TiB 77.52 1.07 67 up 119 hdd 9.09470 1.00000 9.1 TiB 5.8 TiB 5.8 TiB 64 KiB 16 GiB 3.3 TiB 64.05 0.88 62 up 120 hdd 9.09470 1.00000 9.1 TiB 7.2 TiB 7.2 TiB 60 KiB 19 GiB 1.9 TiB 79.13 1.09 75 up 122 hdd 9.09470 1.00000 9.1 TiB 6.4 TiB 6.4 TiB 68 KiB 18 GiB 2.7 TiB 70.19 0.97 69 up 125 hdd 9.09470 1.00000 9.1 TiB 7.3 TiB 7.3 TiB 36 KiB 19 GiB 1.8 TiB 80.11 1.10 75 up 127 hdd 9.09470 1.00000 9.1 TiB 7.8 TiB 7.8 TiB 3.7 MiB 20 GiB 1.3 TiB 85.56 1.18 82 up 129 hdd 9.09470 1.00000 9.1 TiB 7.0 TiB 7.0 TiB 60 KiB 19 GiB 2.1 TiB 77.27 1.07 73 up 131 hdd 9.09470 1.00000 9.1 TiB 7.0 TiB 7.0 TiB 80 KiB 18 GiB 2.1 TiB 77.25 1.07 74 up 133 hdd 9.09470 1.00000 9.1 TiB 6.6 TiB 6.6 TiB 84 KiB 18 GiB 2.5 TiB 73.06 1.01 67 up 135 hdd 9.09470 1.00000 9.1 TiB 6.5 TiB 6.5 TiB 92 KiB 17 GiB 2.6 TiB 71.44 0.99 69 up 92 hdd 9.09470 1.00000 9.1 TiB 6.5 TiB 6.5 TiB 84 KiB 17 GiB 2.6 TiB 71.93 0.99 71 up 93 hdd 9.09470 1.00000 9.1 TiB 6.7 TiB 6.7 TiB 104 KiB 17 GiB 2.4 TiB 73.58 1.01 71 up 94 hdd 9.09470 1.00000 9.1 TiB 5.4 TiB 5.4 TiB 92 KiB 15 GiB 3.7 TiB 59.80 0.82 57 up 95 hdd 9.09470 1.00000 9.1 TiB 7.2 TiB 7.2 TiB 32 KiB 19 GiB 1.9 TiB 79.08 1.09 76 up 96 hdd 9.09470 1.00000 9.1 TiB 5.7 TiB 5.7 TiB 88 KiB 15 GiB 3.3 TiB 63.18 0.87 61 up 97 hdd 9.09470 1.00000 9.1 TiB 6.9 TiB 6.9 TiB 144 KiB 18 GiB 2.2 TiB 75.69 1.04 74 up 98 hdd 9.09470 1.00000 9.1 TiB 5.6 TiB 5.6 TiB 128 KiB 15 GiB 3.5 TiB 62.05 0.86 61 up 99 hdd 9.09470 1.00000 9.1 TiB 7.2 TiB 7.2 TiB 72 KiB 19 GiB 1.9 TiB 78.85 1.09 76 up 0 ssd 3.49219 1.00000 3.5 TiB 808 GiB 768 GiB 36 GiB 3.7 GiB 2.7 TiB 22.59 0.31 147 up 1 ssd 3.49219 1.00000 3.5 TiB 815 GiB 762 GiB 49 GiB 3.6 GiB 2.7 TiB 22.79 0.31 146 up 2 ssd 3.49219 1.00000 3.5 TiB 828 GiB 774 GiB 51 GiB 3.8 GiB 2.7 TiB 23.17 0.32 150 up 3 ssd 3.49219 1.00000 3.5 TiB 805 GiB 768 GiB 33 GiB 3.5 GiB 2.7 TiB 22.51 0.31 146 up 4 ssd 3.49219 1.00000 3.5 TiB 849 GiB 810 GiB 34 GiB 4.6 GiB 2.7 TiB 23.74 0.33 156 up 6 ssd 3.49219 1.00000 3.5 TiB 800 GiB 762 GiB 35 GiB 3.6 GiB 2.7 TiB 22.38 0.31 146 up 8 ssd 3.49219 1.00000 3.5 TiB 886 GiB 843 GiB 40 GiB 3.4 GiB 2.6 TiB 24.77 0.34 156 up 10 ssd 3.49219 1.00000 3.5 TiB 820 GiB 774 GiB 43 GiB 3.2 GiB 2.7 TiB 22.93 0.32 146 up 5 ssd 3.49219 1.00000 3.5 TiB 755 GiB 719 GiB 32 GiB 4.2 GiB 2.8 TiB 21.12 0.29 141 up 7 ssd 3.49219 1.00000 3.5 TiB 741 GiB 698 GiB 39 GiB 3.6 GiB 2.8 TiB 20.71 0.29 136 up 9 ssd 3.49219 1.00000 3.5 TiB 711 GiB 665 GiB 42 GiB 3.8 GiB 2.8 TiB 19.88 0.27 131 up 11 ssd 3.49219 1.00000 3.5 TiB 815 GiB 791 GiB 22 GiB 2.4 GiB 2.7 TiB 22.79 0.31 138 up 101 ssd 3.49219 1.00000 3.5 TiB 848 GiB 805 GiB 38 GiB 3.7 GiB 2.7 TiB 23.70 0.33 146 up 103 ssd 3.49219 1.00000 3.5 TiB 860 GiB 816 GiB 40 GiB 3.4 GiB 2.7 TiB 24.04 0.33 150 up 105 ssd 3.49219 1.00000 3.5 TiB 813 GiB 774 GiB 36 GiB 2.5 GiB 2.7 TiB 22.72 0.31 148 up 107 ssd 3.49219 1.00000 3.5 TiB 853 GiB 800 GiB 50 GiB 2.9 GiB 2.7 TiB 23.86 0.33 147 up 100 ssd 3.49219 1.00000 3.5 TiB 814 GiB 765 GiB 47 GiB 2.6 GiB 2.7 TiB 22.78 0.31 144 up 102 ssd 3.49219 1.00000 3.5 TiB 763 GiB 749 GiB 13 GiB 2.1 GiB 2.7 TiB 21.34 0.29 141 up 104 ssd 3.49219 1.00000 3.5 TiB 816 GiB 783 GiB 31 GiB 3.0 GiB 2.7 TiB 22.83 0.31 144 up 106 ssd 3.49219 1.00000 3.5 TiB 803 GiB 734 GiB 66 GiB 3.1 GiB 2.7 TiB 22.45 0.31 143 up 108 ssd 3.49219 1.00000 3.5 TiB 830 GiB 791 GiB 36 GiB 2.5 GiB 2.7 TiB 23.20 0.32 156 up 109 ssd 3.49219 1.00000 3.5 TiB 822 GiB 774 GiB 45 GiB 2.7 GiB 2.7 TiB 22.99 0.32 146 up 110 ssd 3.49219 1.00000 3.5 TiB 873 GiB 842 GiB 28 GiB 2.6 GiB 2.6 TiB 24.40 0.34 154 up 111 ssd 3.49219 1.00000 3.5 TiB 796 GiB 749 GiB 45 GiB 2.6 GiB 2.7 TiB 22.27 0.31 146 up TOTAL 1.1 PiB 799 TiB 796 TiB 930 GiB 2.1 TiB 303 TiB 72.51 MIN/MAX VAR: 0.27/1.24 STDDEV: 22.37 The HDDs are still in rebalance, I had raised PGs to 4096, but will probably need to do again to 8172. But let's see first if it helps a bit at least.
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
