Hey Mohamed,
just send us the output of
ceph -s
and
ceph mon dump
please.
Best,
Malte
On 02.11.23 13:05, Mohamed LAMDAOUAR wrote:
Hi robert,
when I ran this command, I got this error (because the database of the osd
was on the boot disk)
ceph-objectstore-tool \
--type bluestore \
--data-path /var/lib/ceph/c80891ba-55f3-11ed-9389-919f4368965c/osd.9 \
--op update-mon-db \
--mon-store-path /home/enyx-admin/backup-osd-9 \
--no-mon-config --debug
2023-11-02T10:59:33.381+0000 7f6724da71c0 1 bdev(0x560257b42400
/var/lib/ceph/c80891ba-55f3-11ed-9389-919f4368965c/osd.9/block) open path
/var/lib/ceph/c80891ba-55f3-11ed-9389-919f4368965c/osd.9/block
2023-11-02T10:59:33.381+0000 7f6724da71c0 1 bdev(0x560257b42400
/var/lib/ceph/c80891ba-55f3-11ed-9389-919f4368965c/osd.9/block) open size
10000827154432 (0x9187fc00000, 9.1 TiB) block_size 4096 (4 KiB) rotational
device, discard not supported
2023-11-02T10:59:33.381+0000 7f6724da71c0 1 bluestore(/var/lib/ceph/
c80891ba-55f3-11ed-9389-919f4368965c/osd.9) _set_cache_sizes cache_size
1073741824 meta 0.45 kv 0.45 data 0.06
2023-11-02T10:59:33.381+0000 7f6724da71c0 1 bdev(0x560257b42c00
/var/lib/ceph/c80891ba-55f3-11ed-9389-919f4368965c/osd.9/block) open path
/var/lib/ceph/c80891ba-55f3-11ed-9389-919f4368965c/osd.9/block
2023-11-02T10:59:33.381+0000 7f6724da71c0 1 bdev(0x560257b42c00
/var/lib/ceph/c80891ba-55f3-11ed-9389-919f4368965c/osd.9/block) open size
10000827154432 (0x9187fc00000, 9.1 TiB) block_size 4096 (4 KiB) rotational
device, discard not supported
2023-11-02T10:59:33.381+0000 7f6724da71c0 1 bluefs add_block_device bdev 1
path /var/lib/ceph/c80891ba-55f3-11ed-9389-919f4368965c/osd.9/block size
9.1 TiB
2023-11-02T10:59:33.381+0000 7f6724da71c0 1 bluefs mount
2023-11-02T10:59:33.381+0000 7f6724da71c0 1 bluefs _init_alloc shared, id
1, capacity 0x9187fc00000, block size 0x10000
2023-11-02T10:59:33.441+0000 7f6724da71c0 -1 bluefs _replay 0x0: stop: uuid
369c96dd-2df1-8d88-2722-3f8334920e83 != super.uuid
ba94c6e8-394b-4a78-84d6-9afe1cbc280b,
block dump:
00000000 42 9c 61 78 69 ec 36 9c 96 dd 2d f1 8d 88 27 22
|B.axi.6...-...'"|
00000010 3f 83 34 92 0e 83 e6 0f 8f 17 fc 3e ec 86 c5 15
|?.4........>....|
00000020 39 91 13 2e b0 14 92 86 65 75 5c 8e c1 ee fc 18
|9.......eu\.....|
00000030 f1 7b b2 37 f7 75 70 e2 5e da 79 cd e6 ad 27 40
|.{.7.up.^.y...'@|
00000040 d6 b8 3b da 81 1f 9b ba c6 e8 b7 68 bc a1 77 ac
|..;........h..w.|
00000050 7b a9 a3 cd 9d da b6 57 aa 40 bd ab d0 89 ec e6 |{......W.@
......|
00000060 71 a2 2b 4d 87 74 2f ff 0a bf 3b da 3d da 93 52
|q.+M.t/...;.=..R|
00000070 1c ea f2 fb 8d e0 a1 e6 ef b5 42 5e 85 87 27 df
|..........B^..'.|
00000080 ac f1 ae 08 9d c5 71 6f 0f f7 68 ce 28 3d 3e 6e
|......qo..h.(=>n|
00000090 94 b2 1a dc 3b f0 9e e9 6e 77 dd 95 b6 9e 94 56
|....;...nw.....V|
000000a0 f2 dd 9a 35 a0 65 78 05 bb a9 5f a1 99 6a 5c a1
|...5.ex..._..j\.|
000000b0 5d e9 6d 02 83 be 9d 60 d1 82 fc 6c 66 40 11 17 |].m....`...lf@
..|
000000c0 3a 4d 9d 73 f6 ec fb ed 41 db e2 39 15 e1 5f 28
|:M.s....A..9.._(|
000000d0 c4 ce cf eb 93 f2 88 d5 af ae 11 14 d6 97 74 ff
|..............t.|
000000e0 4b 7e 73 fe 97 4c 06 2a 3a bc b3 7f 04 94 6c 1d
|K~s..L.*:.....l.|
000000f0 60 bf b1 42 fa 76 b0 df 33 ff bf 84 36 b1 b5 b3
|`..B.v..3...6...|
00000100 17 36 d6 b7 7d 4c d4 37 fa 7f 8e 59 1f 72 53 d5
|.6..}L.7...Y.rS.|
00000110 c4 d0 de d8 4e 13 ca c6 0a 60 87 3c e4 21 2b 1b
|....N....`.<.!+.|
00000120 00 f2 67 cf 0a 02 01 20 ec ec 7f c1 8f e3 df f8 |..g....
........|
00000130 3f db 7f 60 28 14 8a fa 48 cb c6 f6 c7 9a 3f 71
|?..`(...H.....?q|
00000140 bf 61 36 30 08 c0 f1 e7 f8 af b5 7f d2 fc ad a1
|.a60............|
00000150 72 b2 40 ff 82 ff a3 c7 5f f0 a3 0e 8f b2 fe b6 |r.@
....._.......|
00000160 ee 2f 5d fe 90 8b fa 28 8f 95 03 fa 5b ee e3 9c
|./]....(....[...|
00000170 36 ea 3f 6a 1e c0 fe bb c2 80 4a 56 ca 96 26 8f
|6.?j......JV..&.|
00000180 85 03 e0 f8 67 c9 3d a8 fa 97 af c5 c0 00 ce 7f
|....g.=.........|
00000190 cd 83 ff 36 ff c0 1c f0 7b c1 03 cf b7 b6 56 06
|...6....{.....V.|
000001a0 8a 30 7b 4d e0 5b 11 31 a0 12 cc d9 5e fb 7f 2d
|.0{M.[.1....^..-|
000001b0 fb 47 04 df ea 1b 3d 3e 6c 1f f7 07 96 df 97 cf
|.G....=>l.......|
000001c0 15 60 76 56 0e b6 06 30 3b c0 6f 11 0a 40 19 98 |.`vV...0;.o..@
..|
000001d0 a1 89 fe e3 b6 f3 28 80 83 e5 c1 1d 9c ac da 40
|......(........@|
000001e0 71 5b 2b 07 eb 07 2e 8a 0f 71 7f 88 aa f5 23 0b
|q[+......q....#.|
000001f0 03 17 a0 b0 e2 c3 76 e3 68 62 00 53 10 17 02 4a
|......v.hb.S...J|
00000200 02 ec 1f 72 82 8f 0f 28 fc a0 e0 83 04 3b c0 e3
|...r...(.....;..|
00000210 6b 25 15 fe ae ce df de 33 29 6c e5 f0 68 c6 1f
|k%......3)l..h..|
00000220 e9 01 00 ff f1 be c9 c7 f4 f8 73 fc bf cc 80 fe
|..........s.....|
00000230 73 1d 08 a8 64 62 6f 0e e3 11 13 15 13 03 81 58
|s...dbo........X|
00000240 a1 20 10 9b f0 43 e3 7c 68 2c 0f ed 21 34 10 10 |.
...C.|h,..!4..|
00000250 08 04 05 f3 fd de 0f ed 35 ff b0 4d 4d 5d e3 a1
|........5..MM]..|
00000260 7f 30 08 f0 b0 85 fe e9 06 f0 3f 95 64 f9 2f 3e
|.0........?.d./>|
00000270 7b 48 e3 7f 3a 0a f5 7f 3c 05 f8 9b ab 1e a3 2f
|{H..:...<....../|
00000280 6d 62 68 a7 09 f8 4d f5 08 fe 69 e9 df ed f9 a7
|mbh...M...i.....|
00000290 0c 61 fd 87 fd dc ca e8 0f 59 7f 44 06 15 28 67
|.a.......Y.D..(g|
000002a0 0d b3 14 34 78 2c a4 3f d4 07 fc d6 df d2 c1 dc
|...4x,.?........|
000002b0 fc 8f 07 e8 81 b7 9c 83 bd b9 89 e5 a3 52 90 7f
|.............R..|
000002c0 39 ed 6f ff 37 20 fc 50 d7 0f 89 f8 a7 bb 58 1e |9.o.7
.P......X.|
000002d0 5c c4 fe e0 9f 87 02 01 b1 b3 f3 fd 4e 22 43 07
|\...........N"C.|
000002e0 03 d8 5f f3 ff 71 25 e4 4f ba 07 77 b2 3c e0 d9
|.._..q%.O..w.<..|
000002f0 d8 fe 6c 8f 73 d0 bf b9 f9 b1 17 7d e8 41 0f bc
|..l.s......}.A..|
00000300 7e 8b 7a d0 59 e4 e1 98 a0 17 e1 82 80 c0 ec 8f
|~.z.Y...........|
00000310 8c c0 ec 0f c2 a0 0c 20 08 1d 08 44 f7 ea 6f 6a |.......
...D..oj|
00000320 3a db c2 de a3 82 00 0f 07 28 e8 af 0f 80 8d 95
|:........(......|
00000330 15 ca 0a 78 0f f8 03 07 e1 04 b3 3d 64 c0 ef 19
|...x.......=d...|
00000340 4b c0 5f 74 60 ce 7f e0 c0 50 30 2b cb 3f 70 10
|K._t`....P0+.?p.|
00000350 ce 87 c3 f9 df 74 ec 6c ff a4 63 e7 7c 28 dd 7f
|.....t.l..c.|(..|
00000360 e1 38 ff 29 03 c2 01 e6 fc a7 2e 10 0e 76 f0 bf
|.8.).........v..|
00000370 70 9c a0 7f cb 78 38 82 39 fe 85 83 82 fe 4d 07
|p....x8.9.....M.|
00000380 65 83 fc 0b c7 ca 0e fa 17 8e ed ef b6 d9 db ea
|e...............|
00000390 3f 9c e7 b6 8f f1 56 34 71 85 3d b8 14 a8 60 65
|?.....V4q.=...`e|
000003a0 f5 90 79 7f e6 93 a4 e5 7b 2b 00 98 f5 4f 40 04 |..y.....{+...O@
.|
000003b0 a0 09 e0 01 83 a0 1c ec 42 42 ac 60 56 56 41 36
|........BB.`VVA6|
000003c0 41 11 11 56 61 28 04 c2 21 26 c6 01 15 61 85 40
|A..Va(..!&...a.@|
000003d0 85 20 7c a8 ff 37 c9 63 8a 8b 58 19 08 1b c3 0c |.
|..7.c..X.....|
000003e0 cc ec 1c 2c 00 40 31 0e 30 94 45 94 93 0d c4 c9
|...,.@1.0.E.....|
000003f0 c9 ce 21 04 82 b0 41 39 45 20 42 ac 10 21 31 0e |..!...A9E
B..!1.|
00000400 08 54 84 83 13 f5 f7 f7 06 7d 5b fb df a1 87 70
|.T.......}[....p|
00000410 72 40 39 50 69 69 45 e5 c4 50 ff 1f ce 58 4b a0 |r@9PiiE.
.P...XK.|
00000420 b8 3b 78 9c 6d 53 41 8e db 30 0c bc eb 15 04 7a
|.;x.mSA..0.....z|
00000430 69 17 8d 1f 90 9c da 5d b4 dd 5b 80 4d 1f 40 4b
|i......]..[.M.@K|
00000440 b4 c3 b5 2c 1a 92 6c 63 7f 5f 4a 89 03 24 5b 9f
|...,..lc._J..$[.|
00000450 6c 93 33 1c cd 50 4d 03 11 d7 fd 1e 26 d7 19 03
|l.3..PM.....&...|
00000460 00 47 ec e9 67 24 1c 40 02 3d 8b 9f c7 60 cc 97
|.G..g$.@.=...`..|
00000470 bb c7 bc 86 1c c5 cd 36 b3 84 c7 9a 39 9d 39 81
|.......6....9.9.|
00000480 13 3b 8f 14 32 38 4a 36 72 4b 09 f2 99 e0 d7 f1
|.;..28J6rK......|
00000490 f7 0f e8 38 8e 2b 46 02 0c 0e 92 74 b9 7e f0 38
|...8.+F....t.~.8|
000004a0 79 2a 10 72 80 09 26 8c 19 a4 ab a8 a7 13 db 61
|y*.r..&........a|
000004b0 97 65 77 8a e8 08 a6 28 ef 64 f3 53 63 8c 79 d9
|.ew....(.d.Sc.y.|
000004c0 e6 bc 59 99 e8 93 96 b7 89 2c a3 07 cc 59 9b 54
|..Y......,...Y.T|
000004d0 2d ac ec 3d b4 04 3d 2f 14 20 4b 1d 70 53 f4 7a |-..=..=/.
K.pS.z|
000004e0 04 2b 51 c5 7e c5 01 e1 85 12 f7 01 fe 06 ce e9
|.+Q.~...........|
000004f0 1b 38 8e 3a d5 7f 00 87 45 fc a2 32 39 5c d0 98
|.8.:....E..29\..|
00000500 32 74 44 ae 45 3b 18 2f 32 7d 57 96 10 b4 9b 43
|2tD.E;./2}W....C|
00000510 5f 5b 46 8c 03 a9 19 98 11 02 e5 55 e2 a0 e8 4c
|_[F........U...L|
00000520 b1 43 4b 45 85 4e 00 89 8e 22 a8 ce f8 a1 f8 b9
|.CKE.N..."......|
00000530 94 8b 0b cd a3 a5 19 63 4f da 8f 01 70 76 4c 41
|.......cO...pvLA|
00000540 29 56 ce 67 68 31 b1 85 39 28 4d ca 6a 6e 99 2e
|)V.gh1..9(M.jn..|
00000550 57 8d e2 bd ac 55 8e 4c 6c d3 7e 93 a1 bf 0e 90
|W....U.Ll.~.....|
00000560 b2 06 3e 96 6a 51 b8 eb b4 f5 60 4a 3a 23 8d a2
|..>.jQ....`J:#..|
00000570 6a 46 9c a6 7a de 8b 62 6d bc f7 9e ea 26 a4 07
|jF..z..bm....&..|
00000580 fb ff b3 0d e5 9d 17 76 9b 79 91 08 d2 15 bd 37
|.......v.y.....7|
00000590 66 57 ce 24 0b c5 85 69 dd 0a 25 ef 02 49 d0 53
|fW.$...i..%..I.S|
000005a0 a0 a8 69 72 e8 24 8e 58 8b d7 f3 f5 82 3e 5d f6
|..ir.$.X.....>].|
000005b0 45 67 5c 37 e4 50 08 6f 34 8e 32 6a f8 5b 24 f7
|Eg\7.P.o4.2j.[$.|
000005c0 ab 78 db be ca 5a 81 7a fc 8e 03 6a e0 7a 11 6e
|.x...Z.z...j.z.n|
000005d0 2c b6 c8 db 38 b6 e5 bd 73 e3 0f a7 ac a6 7d 32
|,...8...s.....}2|
000005e0 c3 98 e7 33 86 be dc 04 b9 c8 bc d9 52 14 78 85
|...3........R.x.|
000005f0 5d 6c 69 49 fd 57 ca a6 d1 2f eb 67 47 7a 3d 6d
|]liI.W.../.gGz=m|
00000600 c5 7a e9 1b cd 56 c7 fd 03 f1 ee 40 a1 e5 06 82 |.z...V.....@
....|
00000610 6d 78 9c db c1 de cf 3e a1 66 63 e7 4f 46 c1 dc
|mx.....>.fc.OF..|
00000620 c4 cc 3c 85 e2 d4 e4 92 cc fc bc 62 3d 2e 2e c7
|..<........b=...|
00000630 c9 13 98 9c 6c 52 15 92 f3 8b 8a 52 8b 0b f2 f3
|....lR.....R....|
00000640 52 32 f3 d2 15 0a 8a f2 b3 80 6a f4 14 1c f3 f2
|R2........j.....|
00000650 4b 32 52 8b 60 3a 14 32 8b b9 52 52 53 32 93 13
|K2R.`:.2..RRS2..|
00000660 4b 52 53 14 4a f2 15 80 92 93 ff 32 49 4c 76 65
|KRS.J......2ILve|
00000670 2e 06 00 81 82 25 40 b8 c5 05 78 9c b5 7a 65 54
|.....%@...x..zeT|
00000680 5c cb d6 6d e3 d2 48 70 97 20 01 82 5b 20 b8 bb |\..m..Hp. ..[
..|
00000690 43 70 f7 c6 a1 1b 97 20 01 02 04 08 0e 21 c1 5d |Cp.....
.....!.]|
000006a0 1a 02 04 0d ee ee ee 6e c1 dd 5f 72 df 77 c6 77
|.......n.._r.w.w|
000006b0 72 de 39 77 dc fb e3 55 8f ea b1 ab 47 cd 59 ab
|r.9w...U....G.Y.|
000006c0 6b ed 3d 6b ed aa a5 22 0f 07 8f 0b 00 20 03 00 |k.=k...".....
..|
000006d0 bd 22 3d 72 e9 70 2c 1d 8c 00 00 e0 57 fd f9 13
|."=r.p,.....W...|
000006e0 c0 1e 64 6f 01 f1 74 b2 30 71 72 b2 03 99 99 40
|..do..t.0qr....@|
000006f0 40 8e 0e ac 6e 0e e6 2c 8e 26 60 10 98 c5 d1 c9
|@...n..,.&`.....|
00000700 c2 c1 dc d1 cc d5 de c2 01 c2 62 e5 62 e2 64 0d
|..........b.b.d.|
00000710 32 03 ab fc 46 68 ac 3a d7 61 00 04 00 7e 55 c2
|2...Fh.:.a...~U.|
00000720 9f 84 1a d6 ae f6 a6 0e 26 20 3b 30 2b e4 8f 4b |........&
;0+..K|
00000730 16 27 07 ab 30 15 25 69 74 54 62 d4 9f 5d d0 65
|.'..0.%itTb..].e|
00000740 65 24 d4 00 00 98 9f 97 80 6f c8 70 3f bf 43 17
|e$.......o.p?.C.|
00000750 46 5f 00 00 b0 f0 2a 0a 1a 92 3f 9b 40 20 10 0f |F_....*...?.@
..|
00000760 0f 8f 8c 8c 8c 94 9c 92 86 86 96 85 85 85 96 ed
|................|
00000770 25 b5 00 35 37 37 37 07 f7 6b 21 21 a1 17 dc 92
|%..5777..k!!....|
00000780 f4 e2 f4 cc 0a cc 9c 6f 38 5f e9 bd e2 35 e4 e5
|.......o8_...5..|
00000790 37 e3 67 15 d3 e4 90 d2 e3 94 31 e4 96 37 e3 55
|7.g.......1..7.U|
000007a0 b2 7c ad 6c cd a7 6a 23 a4 e5 ac c5 a5 a5 cb ad
|.|.l..j#........|
000007b0 6b c4 6b 64 ca 67 aa a4 a4 a4 a1 a1 a1 a7 a7 27
|k.kd.g.........'|
000007c0 a2 eb 2a 6e e8 2d 6a 27 2a 09 91 94 f3 95 93 34
|..*n.-j'*......4|
000007d0 f1 73 10 75 80 48 42 7c e5 7d cd cc cc ec ed ed
|.s.u.HB|.}......|
000007e0 2d ed 20 0e ae be 76 3e 3e 60 30 d8 db db 5b ca |-.
...v>>`0...[.|
000007f0 34 40 c6 e2 9d 9c 75 a8 82 4d 84 92 c3 47 15 c7 |4@
....u..M...G..|
00000800 58 35 70 a2 86 5b 8a a6 c7 17 6d ef 4c a5 20 25 |X5p..[....m.L.
%|
00000810 95 60 15 b5 70 35 cd 8f 9a 3a 09 3a fa a9 fa 46
|.`..p5...:.:...F|
00000820 19 46 26 d9 26 16 45 16 d6 50 6b db 4a 5b 87 5a
|.F&.&.E..Pk.J[.Z|
00000830 07 e7 46 67 48 3b c4 a3 db c3 b3 df d3 77 d4 d7
|..FgH;.......w..|
00000840 20 a8 d8 30 e4 ab 71 08 d4 38 b4 dc 2c bc ca 22 |
..0..q..8..,.."|
00000850 aa de ea e3 77 eb 98 46 8f 94 66 e3 c8 16 ab 84
|....w..F..f.....|
00000860 6e eb f8 4e fb e4 2e 87 4f bd ce 69 43 41 4a 41
|n..N....O..iCAJA|
00000870 a1 2a a1 1f 35 3f 26 e8 24 7c 36 fc 9c 65 92 55
|.*..5?&.$|6..e.U|
00000880 6c 59 0c b5 86 56 da 56 d6 39 d6 35 3a 37 b6 41
|lY...V.V.9.5:7.A|
00000890 da fa 3c fb c6 7c c7 02 02 02 22 22 22 e2 e3 e3
|..<..|...."""...|
000008a0 53 53 53 df 7d ca c8 ce ce 2e 2e 2e ae ac ac 6c
|SSS.}..........l|
000008b0 68 68 08 ce 1f 0a 2f 1e cd 89 69 81 e6 8f e5 54
|hh..../...i....T|
000008c0 f6 e6 7f 6f 2b 6c ec cd 6d 9c 6b 2c 9a ad a8 1f
|...o+l..m.k,....|
000008d0 ae ae 1a ff 56 3f ea 3f e5 1f 34 1f 14 b8 98 1a
|....V?.?..4.....|
000008e0 3e 9e 17 b2 1a 12 b6 19 16 f9 23 32 fa 38 3a e6
|>.........#2.8:.|
000008f0 2c 26 f1 2e 31 f9 31 b9 a4 b5 e7 6b 5b 7f 7d c7
|,&..1.1....k[.}.|
00000900 58 6d f7 5c 63 ef 6c c3 c0 5a 5b fa d2 a4 ff e4
|Xm.\c.l..Z[.....|
00000910 c2 bb 85 fe a8 ed d5 90 d5 8d f7 1b fb 91 fb 67
|...............g|
00000920 31 67 b7 09 b7 8f c9 8f ed ed ed fd fd fd 2d 3d
|1g............-=|
00000930 e3 ed c3 cb e3 e3 e3 d3 b3 f3 0b 0b 0b 6d c3 2b
|.............m.+|
00000940 9d a3 ab 7d 53 5b f3 eb 87 1b 1b 1b 5b 3f ce 8e
|...}S[......[?..|
00000950 ce ae ce 2e 6f 1e 1f 1f 7f 7a ff e9 e9 49 da bd
|....o....z...I..|
00000960 0d 1b 00 40 c1 95 95 10 d5 f0 98 3b f8 f2 a0 35
|...@.......;...5|
00000970 3f 45 d4 dd 08 91 4d fa a6 23 5d 1d d2 be 0b 12
|?E....M..#].....|
00000980 2f c0 8a 54 e5 ac e3 d2 41 a6 7f a5 f9 2e d8 2e
|/..T....A.......|
00000990 51 1c c3 40 9f ff f5 bb 10 22 53 d9 b8 dc 1f cc |Q..@
....."S.....|
000009a0 b6 af bc bf d9 65 26 8f 74 f2 8b 30 f1 4b 81 b6
|.....e&.t..0.K..|
000009b0 68 ee 93 e2 ee 61 09 8f 5b 1c 74 07 26 cf 26 1b
|h....a..[.t.&.&.|
000009c0 77 12 28 0e 43 b7 85 ae 14 a3 9d ec 66 f4 31 00
|w.(.C.......f.1.|
000009d0 7f 2d 2d 24 d3 fb af de 61 fb 77 cb 97 ee 97 95
|.--$....a.w.....|
000009e0 61 c1 1e 0f 58 e7 a9 8d f3 77 6a c9 c0 3e 43 50
|a...X....wj..>CP|
000009f0 e2 91 37 78 49 a4 df 3b 87 15 82 c5 f9 52 fa c5
|..7xI..;.....R..|
00000a00 87 40 95 39 58 35 89 61 d8 3e 4b a2 41 58 02 e4
|.@.9X5.a.>K.AX..|
00000a10 51 df a8 8f c3 e8 a8 81 2b e4 5c 68 9f d7 07 24
|Q.......+.\h...$|
00000a20 6a 5d f5 a9 9c 44 59 b4 d2 7f 75 c1 8b a8 01 d4
|j]...DY...u.....|
00000a30 46 8a f2 b0 ae d6 8a 12 2e e8 b4 53 f2 12 6d 9b
|F..........S..m.|
00000a40 63 4c 3b a7 e0 38 11 99 00 cb 3b 80 9f 49 71 6a
|cL;..8....;..Iqj|
00000a50 93 29 53 01 12 42 21 a9 00 8f 2f 2b a6 0e 31 fc
|.)S..B!.../+..1.|
00000a60 f9 0d af 95 c4 16 90 08 2f fa 46 e8 c7 d9 7f 7c
|......../.F....||
00000a70 57 95 f8 86 47 7f 84 90 ad 31 e7 8a 2e 08 f4 30
|W...G....1.....0|
00000a80 7f 79 57 00 b5 13 3c d0 bc 54 c5 97 05 5e 27 b9
|.yW...<..T...^'.|
00000a90 19 fb 06 65 21 94 d4 a0 ae 7c 5b 84 c9 b0 85 81
|...e!....|[.....|
00000aa0 64 d5 5f d1 d9 12 8c c5 a7 14 54 59 be e0 d7 f6
|d._.......TY....|
00000ab0 97 33 44 57 a0 f1 89 4f b2 f4 69 02 6c c9 da f6
|.3DW...O..i.l...|
00000ac0 cd bf dc 84 dc 4e a8 33 e8 69 03 b8 b7 34 64 94
|.....N.3.i...4d.|
00000ad0 f7 74 09 14 11 71 c7 e4 11 a4 e0 66 08 5f 87 c7
|.t...q.....f._..|
00000ae0 34 3d c6 33 33 f9 4f e0 a6 27 99 ef b0 b0 1e 64
|4=.33.O..'.....d|
00000af0 83 58 13 46 70 1d 09 82 22 39 0c cc f4 00 7d c8
|.X.Fp..."9....}.|
00000b00 7a 22 5d 25 e7 45 45 b9 63 e1 9f 34 ab 3e d8 bc
|z"]%.EE.c..4.>..|
00000b10 0e 80 d0 16 8e a0 4d c1 c4 ab 22 18 3f f6 ea 30
|......M...".?..0|
00000b20 67 d0 e2 16 c3 64 70 47 be 29 39 7d 41 f4 c2 3e
|g....dpG.)9}A..>|
00000b30 ce 92 4d 1c 71 79 fc b3 65 a3 f0 e0 87 ca e4 65
|..M.qy..e......e|
00000b40 65 04 10 0e 5c 82 4a 2f 0a 5f eb 35 83 9b 80 7f
|e...\.J/._.5....|
00000b50 2d 4e c7 f7 26 67 56 59 24 e9 38 05 e1 3c d3 69
|-N..&gVY$.8..<.i|
00000b60 19 d6 42 6b dc e0 14 09 81 ad ba e9 a4 b9 12 58
|..Bk...........X|
00000b70 d6 18 f8 95 7d 2d 25 5a 41 08 80 5a 7d c8 2f 38
|....}-%ZA..Z}./8|
00000b80 c7 e7 31 a3 48 b5 17 25 5c 03 27 79 8c 42 e1 e3
|..1.H..%\.'y.B..|
00000b90 7b 3d 91 17 19 ec 36 62 bd f0 b1 4c fc 58 32 79
|{=....6b...L.X2y|
00000ba0 1c 4c c5 a0 c2 0f f8 2a 84 c1 16 ec b9 7b 68 4d
|.L.....*.....{hM|
00000bb0 0c 65 f3 ce 4c 2d 23 bd 07 4a c7 b9 07 80 16 eb
|.e..L-#..J......|
00000bc0 f2 af 14 fe 0b ef 7a 82 4a 53 32 7e 6c bd 1b 09
|......z.JS2~l...|
00000bd0 5b 9e 50 aa 4c 75 57 10 94 f8 8a fa a3 74 fd 6b
|[.P.LuW......t.k|
00000be0 1e a3 b8 8c 52 78 e2 01 46 06 46 66 1a 09 0c d7
|....Rx..F.Ff....|
00000bf0 8a 68 11 aa 80 91 ae ae ae a1 6e 05 7e d8 7e 5d
|.h........n.~.~]|
00000c00 5e ba 61 c1 02 bb ba 73 f7 de b1 78 3c 53 46 5d
|^.a....s...x<SF]|
00000c10 14 90 e4 4b 56 3b cc f1 2b 47 30 c0 ad c6 7f 46
|...KV;..+G0....F|
00000c20 59 49 e6 e0 e9 b2 ff 68 6e 22 22 0a a6 85 9f 74
|YI.....hn""....t|
00000c30 f3 f2 ed 5d f0 68 5e 22 1f a2 c8 ac b3 4b fb 14
|...].h^".....K..|
00000c40 e0 95 52 57 13 81 ff 69 94 5f 27 cc d9 fa 10 4c
|..RW...i._'....L|
00000c50 0b 51 75 b9 bf e3 d3 84 82 11 57 23 f8 e9 a2 15
|.Qu.......W#....|
00000c60 e6 7e 53 18 69 5a 18 13 ae 65 99 ba 94 f1 7c 6f
|.~S.iZ...e....|o|
00000c70 05 d0 34 44 a1 8f ef df 2d 0d d3 32 a8 85 74 05
|..4D....-..2..t.|
00000c80 53 0a bb 03 00 d6 b2 e3 dd 66 f9 91 a6 41 f5 67
|S........f...A.g|
00000c90 4f 24 6a 99 81 da bd 65 50 a6 81 e1 ed 33 5b 20
|O$j....eP....3[ |
00000ca0 e3 d4 cd fd f8 a4 8f df 85 ac 15 cc 75 85 a3 b9
|............u...|
00000cb0 75 ed f6 9a b9 b9 cb ca 98 88 c7 4b 01 6e 17 cb
|u..........K.n..|
00000cc0 db 5b c8 95 79 81 2b 70 06 a1 e5 9b 8b b7 6b eb
|.[..y.+p......k.|
00000cd0 26 07 e6 94 65 b6 ed 28 9d 3b 81 83 70 8a 5e 2d
|&...e..(.;..p.^-|
00000ce0 f7 b4 ea 9b 95 7e 77 6f 2a c1 c5 17 fe cb 17 06
|.....~wo*.......|
00000cf0 e4 e4 7e 57 7d 56 14 ee 46 44 e4 7e 6f 8b b6 d1
|..~W}V..FD.~o...|
00000d00 01 bc 4f ad 7b 7d 64 d3 a5 3e 7e 87 51 75 75 96
|..O.{}d..>~.Quu.|
00000d10 43 db 96 8e a5 83 c3 03 25 10 7f 84 85 b2 27 64
|C.......%.....'d|
00000d20 d7 2b 45 37 3d c0 80 f0 63 0a 27 c5 93 97 4b 0b
|.+E7=...c.'...K.|
00000d30 df 6e d4 79 95 21 82 db 0d 60 ba 88 91 c7 4c b7
|.n.y.!...`....L.|
00000d40 65 ce c7 6f 0b 93 62 70 c9 4d 07 80 6f 79 c4 05
|e..o..bp.M..oy..|
00000d50 4b 4f 5e 81 a0 ff e1 fc 02 1d 4d 59 f2 05 ab 21
|KO^.......MY...!|
00000d60 e9 6e dd a8 e0 c3 7c 66 6f e6 30 12 15 dc 3c 03
|.n....|fo.0...<.|
00000d70 d9 04 ec 5a 63 9b 1b 60 f4 7e 6f 41 8c cb 46 f3
|...Zc..`.~oA..F.|
00000d80 d1 1a 72 70 d8 e7 03 b1 3f 5c d2 3f 5c 6a d3 cd
|..rp....?\.?\j..|
00000d90 7f dd fa c0 7d 31 d4 74 7b c3 4f a7 9c e6 c4 bb
|....}1.t{.O.....|
00000da0 44 7c 0f 06 96 b6 c4 a2 18 df d0 03 6a 57 a3 0e
|D|..........jW..|
00000db0 f2 95 9d 49 85 f7 9d ce c3 bd b9 82 ea 86 1c 10
|...I............|
00000dc0 c5 7d 17 db c5 db 63 f6 d3 0d c8 91 23 df e5 41
|.}....c.....#..A|
00000dd0 43 a5 e6 f6 91 08 43 06 12 dc 2e e1 f0 85 3e 2c
|C.....C.......>,|
00000de0 f5 4b 6a 1d 6d a6 0d 6d 9f 9e d9 d6 d1 91 cd 60
|.Kj.m..m.......`|
00000df0 71 c0 e8 93 13 87 9d bf a3 8e 49 53 19 be 9b f6
|q.........IS....|
00000e00 ce bb 9d 83 57 59 ca a6 db e9 7b 5b b5 57 9f 70
|....WY....{[.W.p|
00000e10 67 6e b9 19 94 f2 79 df c7 41 0b 6f 07 19 58 ee
|gn....y..A.o..X.|
00000e20 e6 3b 1e 03 1a bf 73 97 f6 e3 76 69 54 04 78 bb
|.;....s...viT.x.|
00000e30 06 3b a6 22 eb 43 f3 2d f9 8e c2 59 6a dd b4 02
|.;.".C.-...Yj...|
00000e40 d6 4c e0 6a 29 ea b9 54 56 8f 6f 2c e4 e1 62 fa
|.L.j)..TV.o,..b.|
00000e50 93 a0 12 4b 32 ca e5 59 fc 37 b3 f7 3f 16 eb ce
|...K2..Y.7..?...|
00000e60 ce c0 14 4b 0d 77 93 76 f9 d7 fe a5 f8 d0 ba c3
|...K.w.v........|
00000e70 1a 76 7c 6b 0e c2 4f 5f ee 5e f2 f1 54 ba 33 8e
|.v|k..O_.^..T.3.|
00000e80 ac 06 65 12 07 07 e9 b6 8c 92 a2 1d 38 a2 f1 c4
|..e.........8...|
00000e90 10 63 84 b6 ad a6 4e 32 ab cd b6 6d dd 0c a6 ae
|.c....N2...m....|
00000ea0 aa 58 7b bc d4 50 e7 c5 85 0a 6a f8 ae 5d 31 03
|.X{..P....j..]1.|
00000eb0 ae bb 9b 8b 6f eb 2b 74 40 98 93 0e b8 98 ed 28 |....o.+t@
......(|
00000ec0 8f 22 fb b0 ac 6e 2c d5 89 8c f3 87 b4 33 5f 22
|."...n,......3_"|
00000ed0 a2 e5 7b 00 33 e0 4e cf e6 35 12 8b 2a c0 44 7f
|..{.3.N..5..*.D.|
00000ee0 f5 50 98 66 8f 4e 21 b1 3b 3e c1 7a 82 c1 c5 63
|.P.f.N!.;>.z...c|
00000ef0 3d a4 67 7c 4d 91 3a 27 fa 6c 99 da a3 de 59 b3
|=.g|M.:'.l....Y.|
00000f00 f6 19 e7 02 76 b7 2c 6f b7 7c 48 07 49 60 0b d0
|....v.,o.|H.I`..|
00000f10 85 75 b8 f3 5a 8c 49 89 4b e0 36 9b bf 75 ab 75
|.u..Z.I.K.6..u.u|
00000f20 dc d1 c0 9a 58 e0 a1 48 92 c2 18 76 65 4d cc b1
|....X..H...veM..|
00000f30 95 f9 94 bd dd 9c 12 67 a1 fa e3 00 a2 d9 cd 28
|.......g.......(|
00000f40 4c d5 45 f8 40 f1 94 8f 48 9d c8 ce 3b fa cf 93 |L.E.@
...H...;...|
00000f50 8b b5 2b af 40 3b 9f 37 ba 2c 04 5e 3c fb ec 31
|..+.@;.7.,.^<..1|
00000f60 04 ab 22 24 01 bc 16 85 b1 13 fb 38 10 ce 42 80
|.."$.......8..B.|
00000f70 ea 22 3a cd ac a5 c2 61 ee 6b 10 dd 09 dc 5a fa
|.":....a.k....Z.|
00000f80 48 72 6f fc d5 0c f0 be db eb c2 e0 68 50 03 2c
|Hro.........hP.,|
00000f90 0f f8 24 cf e2 da d6 37 52 59 9a c5 9d 88 ed ce
|..$....7RY......|
00000fa0 db 71 c9 bc d2 3c 9c 98 ce ac 07 92 ef dc 94 c2
|.q...<..........|
00000fb0 b3 d5 4a c3 4b 27 48 67 9d c6 48 e2 cf 47 ed 20
|..J.K'Hg..H..G. |
00000fc0 eb 96 bb bc f5 6f 1b c2 94 21 e5 9f 2e 61 35 c3
|.....o...!...a5.|
00000fd0 04 48 95 d5 ee 76 ba 9b 73 05 e5 bb ef 46 bf 9d
|.H...v..s....F..|
00000fe0 3f 73 c1 97 04 30 7f 67 b2 6f 64 71 d8 2f 60 07
|?s...0.g.odq./`.|
00000ff0 f8 f3 fa 17 47 4c 36 d6 f8 b5 a2 7e 56 d2 c5 9a
|....GL6....~V...|
00001000
2023-11-02T10:59:33.441+0000 7f6724da71c0 1 bluefs mount shared_bdev_used
= 0
2023-11-02T10:59:33.441+0000 7f6724da71c0 1 bluestore(/var/lib/ceph/
c80891ba-55f3-11ed-9389-919f4368965c/osd.9) _prepare_db_environment set
db_paths to db,9500785796710 db.slow,9500785796710
2023-11-02T10:59:33.441+0000 7f6724da71c0 -1 rocksdb: verify_sharding
unable to list column families: NotFound:
2023-11-02T10:59:33.441+0000 7f6724da71c0 -1 bluestore(/var/lib/ceph/
c80891ba-55f3-11ed-9389-919f4368965c/osd.9) _open_db erroring opening db:
2023-11-02T10:59:33.441+0000 7f6724da71c0 1 bluefs umount
2023-11-02T10:59:33.441+0000 7f6724da71c0 1 bdev(0x560257b42c00
/var/lib/ceph/c80891ba-55f3-11ed-9389-919f4368965c/osd.9/block) close
2023-11-02T10:59:33.673+0000 7f6724da71c0 1 bdev(0x560257b42400
/var/lib/ceph/c80891ba-55f3-11ed-9389-919f4368965c/osd.9/block) close
Mount failed with '(5) Input/output error'
8 rue greneta, 75003, Paris, FRANCE
enyx.com
<https://antiphishing.vadesecure.com/v4?f=bWE5cld0MzUzR3BLMGVRdLG7jnjJbjinmRApBW0OzqUbjyQUBC-eQoP-QP_on_dj&i=MDNBanNCVWFVVFptR001Wn8FmozqooQiiVpDAryG6ms&k=yXQk&r=THJuMWVjUHZ1WlRrRll4QwzxCNuHhzLGB-WK0uHgjstxD2ZRhTZ1h7LsTGh6AJfd&s=f60431c0fff3b3cbb45928f29fb98ec224de5d2933de1018be4923bd1edfe281&u=https%3A%2F%2Fspam.exegy.com%2Ffmlurlsvc%2F%3FfewReq%3D%3AB%3AJVMxOj89MS19NjklOy1ibzY7OjE7Oi14Ymxlan9%2BeW42OTxpbz88OTxvMzJobm88am48PT07Pj8%2BaTk5PT09PWhtbT8%2BOWo7PS1%2FNjo9Mzg8Oz08MzwtemJvNjg%2FSjNBTW9lOzg7OzI9Jjg%2FSjNBTW97Ozg7OzI9LXloe382ZW5%2FeG5oS25zbmxyJWhkZi1oNj45LWNvZzY7%26url%3Dhttps%253a%252f%252fwww.enyx.com%252f>
<https://antiphishing.vadesecure.com/v4?f=bWE5cld0MzUzR3BLMGVRdLG7jnjJbjinmRApBW0OzqUbjyQUBC-eQoP-QP_on_dj&i=MDNBanNCVWFVVFptR001Wn8FmozqooQiiVpDAryG6ms&k=yXQk&r=THJuMWVjUHZ1WlRrRll4QwzxCNuHhzLGB-WK0uHgjstxD2ZRhTZ1h7LsTGh6AJfd&s=1a1f6be447b254ad7371210a41df6110af466644c9f408c4eeb5b4d98d7f3267&u=https%3A%2F%2Fspam.exegy.com%2Ffmlurlsvc%2F%3FfewReq%3D%3AB%3AJVMxOj89MS19NjklOy1ibzY7OjE7Oi14Ymxlan9%2BeW42aW8yMzM7b2g8OG5vamg4amhtbm5pbTg8bz9ubTo%2FaTM8bmhqbmk6Pi1%2FNjo9Mzg8Oz08MzwtemJvNjg%2FSjNBTW9lOzg7OzI9Jjg%2FSjNBTW97Ozg7OzI9LXloe382ZW5%2FeG5oS25zbmxyJWhkZi1oNj45LWNvZzY7%26url%3Dhttp%253a%252f%252fenyx.com%252f>
*exegy.com
<https://antiphishing.vadesecure.com/v4?f=bWE5cld0MzUzR3BLMGVRdLG7jnjJbjinmRApBW0OzqUbjyQUBC-eQoP-QP_on_dj&i=MDNBanNCVWFVVFptR001Wn8FmozqooQiiVpDAryG6ms&k=yXQk&r=THJuMWVjUHZ1WlRrRll4QwzxCNuHhzLGB-WK0uHgjstxD2ZRhTZ1h7LsTGh6AJfd&s=9a0739ed5650af623f79b66f75ad57547bede45ec4ea62618c7fb8f73ce9592a&u=https%3A%2F%2Fwww.exegy.com%2F>*
*Mohamed*
*Lamdaouar*
*Infrastructure Engineer*
mohamed.lamdao...@enyx.fr
Le jeu. 2 nov. 2023 à 12:51, Mohamed LAMDAOUAR <mohamed.lamdao...@enyx.fr>
a écrit :
Thanks Robert,
I tried this but I'm stuck. If you have some time, do help me with that I
will be very happy because I'm lost :(
8 rue greneta, 75003, Paris, FRANCE
enyx.com
<https://antiphishing.vadesecure.com/v4?f=bWE5cld0MzUzR3BLMGVRdLG7jnjJbjinmRApBW0OzqUbjyQUBC-eQoP-QP_on_dj&i=MDNBanNCVWFVVFptR001Wn8FmozqooQiiVpDAryG6ms&k=yXQk&r=THJuMWVjUHZ1WlRrRll4QwzxCNuHhzLGB-WK0uHgjstxD2ZRhTZ1h7LsTGh6AJfd&s=f60431c0fff3b3cbb45928f29fb98ec224de5d2933de1018be4923bd1edfe281&u=https%3A%2F%2Fspam.exegy.com%2Ffmlurlsvc%2F%3FfewReq%3D%3AB%3AJVMxOj89MS19NjklOy1ibzY7OjE7Oi14Ymxlan9%2BeW42OTxpbz88OTxvMzJobm88am48PT07Pj8%2BaTk5PT09PWhtbT8%2BOWo7PS1%2FNjo9Mzg8Oz08MzwtemJvNjg%2FSjNBTW9lOzg7OzI9Jjg%2FSjNBTW97Ozg7OzI9LXloe382ZW5%2FeG5oS25zbmxyJWhkZi1oNj45LWNvZzY7%26url%3Dhttps%253a%252f%252fwww.enyx.com%252f>
<https://antiphishing.vadesecure.com/v4?f=bWE5cld0MzUzR3BLMGVRdLG7jnjJbjinmRApBW0OzqUbjyQUBC-eQoP-QP_on_dj&i=MDNBanNCVWFVVFptR001Wn8FmozqooQiiVpDAryG6ms&k=yXQk&r=THJuMWVjUHZ1WlRrRll4QwzxCNuHhzLGB-WK0uHgjstxD2ZRhTZ1h7LsTGh6AJfd&s=1a1f6be447b254ad7371210a41df6110af466644c9f408c4eeb5b4d98d7f3267&u=https%3A%2F%2Fspam.exegy.com%2Ffmlurlsvc%2F%3FfewReq%3D%3AB%3AJVMxOj89MS19NjklOy1ibzY7OjE7Oi14Ymxlan9%2BeW42aW8yMzM7b2g8OG5vamg4amhtbm5pbTg8bz9ubTo%2FaTM8bmhqbmk6Pi1%2FNjo9Mzg8Oz08MzwtemJvNjg%2FSjNBTW9lOzg7OzI9Jjg%2FSjNBTW97Ozg7OzI9LXloe382ZW5%2FeG5oS25zbmxyJWhkZi1oNj45LWNvZzY7%26url%3Dhttp%253a%252f%252fenyx.com%252f>
*exegy.com
<https://antiphishing.vadesecure.com/v4?f=bWE5cld0MzUzR3BLMGVRdLG7jnjJbjinmRApBW0OzqUbjyQUBC-eQoP-QP_on_dj&i=MDNBanNCVWFVVFptR001Wn8FmozqooQiiVpDAryG6ms&k=yXQk&r=THJuMWVjUHZ1WlRrRll4QwzxCNuHhzLGB-WK0uHgjstxD2ZRhTZ1h7LsTGh6AJfd&s=9a0739ed5650af623f79b66f75ad57547bede45ec4ea62618c7fb8f73ce9592a&u=https%3A%2F%2Fwww.exegy.com%2F>*
*Mohamed*
*Lamdaouar*
*Infrastructure Engineer*
mohamed.lamdao...@enyx.fr
Le jeu. 2 nov. 2023 à 12:08, Robert Sander <r.san...@heinlein-support.de>
a écrit :
Hi,
On 11/2/23 11:28, Mohamed LAMDAOUAR wrote:
I have 7 machines on CEPH cluster, the service ceph runs on a docker
container.
Each machine has 4 hdd of data (available) and 2 nvme sssd (bricked)
During a reboot, the ssd bricked on 4 machines, the data are
available on
the HDD disk but the nvme is bricked and the system is not available.
is it
possible to recover the data of the cluster (the data disk are all
available)
You can try to recover the MON db from the OSDs, as they keep a copy of
it:
https://antiphishing.vadesecure.com/v4?f=V3p0eFlQOUZ4czh2enpJS7j2EMloAUIO32fbOvv14lAE8TBfIaEnIXo1udoGNqFP&i=SHV0Y1JZQjNyckJFa3dUQgXEoh5gS3KVt16QIfDo2EM&k=ZVd0&r=T0hnMlUyVEgwNmlmdHc1NSadQBRlJgURlsomDarMY4-jUWYEpAndmhRvBGKG0NP9&s=ce3a282420f5ed2a0e4f461ee9ce796e4a8a9a8a48926d26f05d22ec8b4821cf&u=https%3A%2F%2Fdocs.ceph.com%2Fen%2Freef%2Frados%2Ftroubleshooting%2Ftroubleshooting-mon%2F%23monitor-store-failures
Regards
--
Robert Sander
Heinlein Consulting GmbH
Schwedter Str. 8/9b, 10119 Berlin
https://antiphishing.vadesecure.com/v4?f=V3p0eFlQOUZ4czh2enpJS7j2EMloAUIO32fbOvv14lAE8TBfIaEnIXo1udoGNqFP&i=SHV0Y1JZQjNyckJFa3dUQgXEoh5gS3KVt16QIfDo2EM&k=ZVd0&r=T0hnMlUyVEgwNmlmdHc1NSadQBRlJgURlsomDarMY4-jUWYEpAndmhRvBGKG0NP9&s=9be6bbd716ad7f48663775ee06c8a10f00ee18518820dff2f66261e44d97e44b&u=https%3A%2F%2Fwww.heinlein-support.de
Tel: 030 / 405051-43
Fax: 030 / 405051-19
Amtsgericht Berlin-Charlottenburg - HRB 220009 B
Geschäftsführer: Peer Heinlein - Sitz: Berlin
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
