Dear everyone I can't start osd.21, (attached log file). some pgs can't be repair. I'm using replicate 3 for my data pool. Feel some objects in those pgs be failed,
I tried to delete some data that related above objects, but still not start osd.21
and, removed osd.21, but other osds (eg: osd.86 down, not start osd.86). Guide me to debug it, please! Thanks! -- Tuan Ha Noi - VietNam
2014-10-24 11:10:53.036094 7f86c6fcb780 0 xfsfilestorebackend(/var/lib/ceph/osd/cloud-21) detect_feature: extsize is disabled by conf 2014-10-24 11:10:53.181392 7f86c6fcb780 0 filestore(/var/lib/ceph/osd/cloud-21) mount: WRITEAHEAD journal mode explicitly enabled in conf 2014-10-24 11:10:53.191499 7f86c6fcb780 1 journal _open /dev/sdi1 fd 24: 11998855168 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-10-24 11:11:03.794632 7f86c6fcb780 1 journal _open /dev/sdi1 fd 24: 11998855168 bytes, block size 4096 bytes, directio = 1, aio = 1 2014-10-24 11:11:03.845410 7f86c6fcb780 0 <cls> cls/hello/cls_hello.cc:271: loading cls_hello 2014-10-24 11:11:04.174302 7f86c6fcb780 0 osd.21 101773 crush map has features 1107558400, adjusting msgr requires for clients 2014-10-24 11:11:04.174360 7f86c6fcb780 0 osd.21 101773 crush map has features 1107558400 was 8705, adjusting msgr requires for mons 2014-10-24 11:11:04.174373 7f86c6fcb780 0 osd.21 101773 crush map has features 1107558400, adjusting msgr requires for osds 2014-10-24 11:11:04.174402 7f86c6fcb780 0 osd.21 101773 load_pgs 2014-10-24 11:11:22.986057 7f86c6fcb780 0 osd.21 101773 load_pgs opened 281 pgs 2014-10-24 11:11:23.039971 7f86b6d2e700 0 osd.21 101773 ignoring osdmap until we have initialized 2014-10-24 11:11:23.040818 7f86b6d2e700 0 osd.21 101773 ignoring osdmap until we have initialized 2014-10-24 11:11:23.276236 7f86c6fcb780 0 osd.21 101773 done with init, starting boot process 2014-10-24 11:12:44.346474 7f865ca3c700 0 -- 192.168.1.2:6840/28594 >> 172.30.1.81:0/4234900213 pipe(0x23c15000 sd=66 :6840 s=0 pgs=0 cs=0 l=0 c=0x246f96e0).accept peer addr is really 172.30.1.81:0/4234900213 (socket is 172.30.1.81:47697/0) 2014-10-24 11:15:27.767594 7f86a2505700 -1 *** Caught signal (Segmentation fault) ** in thread 7f86a2505700 ceph version 0.80.7 (6c0127fcb58008793d3c8b62d925bc91963672a3) 1: /usr/bin/ceph-osd() [0x9c830a] 2: (()+0xfcb0) [0x7f86c6009cb0] 3: (ReplicatedPG::trim_object(hobject_t const&)+0x395) [0x8079e5] 4: (ReplicatedPG::TrimmingObjects::react(ReplicatedPG::SnapTrim const&)+0x44c) [0x82215c] 5: (boost::statechart::simple_state<ReplicatedPG::TrimmingObjects, ReplicatedPG::SnapTrimmer, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0xc0) [0x867390] 6: (boost::statechart::state_machine<ReplicatedPG::SnapTrimmer, ReplicatedPG::NotTrimming, std::allocator<void>, boost::statechart::null_exception_translator>::process_queued_events()+0xfb) [0x84d70b] 7: (boost::statechart::state_machine<ReplicatedPG::SnapTrimmer, ReplicatedPG::NotTrimming, std::allocator<void>, boost::statechart::null_exception_translator>::process_event(boost::statechart::event_base const&)+0x1e) [0x84d8de] 8: (ReplicatedPG::snap_trimmer()+0x588) [0x7cc118] 9: (OSD::SnapTrimWQ::_process(PG*)+0x14) [0x675f14] 10: (ThreadPool::worker(ThreadPool::WorkThread*)+0x4e6) [0xa9a366] 11: (ThreadPool::WorkThread::entry()+0x10) [0xa9c380] 12: (()+0x7e9a) [0x7f86c6001e9a] 13: (clone()+0x6d) [0x7f86c4efb31d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- begin dump of recent events --- -10000> 2014-10-24 11:15:20.324218 7f869c4f9700 1 -- 192.168.1.2:0/28594 --> 192.168.1.3:6853/4658 -- osd_ping(ping e101783 stamp 2014-10-24 11:15:20.322565) v2 -- ?+0 0xe955e00 con 0x20af2b00 -9999> 2014-10-24 11:15:20.324268 7f869c4f9700 1 -- 192.168.1.2:0/28594 --> 192.168.1.3:6862/22372 -- osd_ping(ping e101783 stamp 2014-10-24 11:15:20.322565) v2 -- ?+0 0x22e2de00 con 0x20af2840 -9998> 2014-10-24 11:15:20.324313 7f869c4f9700 1 -- 192.168.1.2:0/28594 --> 192.168.1.3:6863/22372 -- osd_ping(ping e101783 stamp 2014-10-24 11:15:20.322565) v2 -- ?+0 0x22e2d700 con 0x20af26e0 -9713> 2014-10-24 11:15:20.365153 7f86ae51d700 5 -- op tracker -- , seq: 18573, time: 2014-10-24 11:15:20.365153, event: done, request: osd_op(client.7869019.0:6944380 rbd_data.451e822ae8944a.0000000000000128 [set-alloc-hint object_size 4194304 write_size 4194304,write 479232~4096] 6.b4cc39f6 snapc 18ee=[18ee] ack+ondisk+write e101783) v4 -9712> 2014-10-24 11:15:20.365266 7f86ae51d700 5 -- op tracker -- , seq: 18576, time: 2014-10-24 11:15:20.365266, event: done, request: osd_sub_op_reply(client.7869019.0:6944380 6.9f6 b4cc39f6/rbd_data.451e822ae8944a.0000000000000128/head//6 [] ondisk, result = 0) v2 -9711> 2014-10-24 11:15:20.378891 7f86b5d2c700 1 -- 192.168.1.2:6841/28594 <== osd.37 192.168.1.3:6851/4658 77 ==== MOSDPGPush(6.4ee 101783 [PushOp(6ad3cee/rbd_data.4cfea248a30bc6.0000000000000102/head//6, version: 101702'47794244, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(6ad3cee/rbd_data.4cfea248a30bc6.0000000000000102/head//6@101702'47794244, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(8b55b4ee/rbd_data.52f42a2ae8944a.000000000000122a/head//6, version: 101714'47794960, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8b55b4ee/rbd_data.52f42a2ae8944a.000000000000122a/head//6@101714'47794960, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 ==== 8390343+0+0 (1363135381 0 0) 0x22059200 con 0x2085d6e0 -2959> 2014-10-24 11:15:26.097850 7f86b5d2c700 5 -- op tracker -- , seq: 18987, time: 2014-10-24 11:15:26.097642, event: dispatched, request: M OSDPGPush(6.872 101783 [PushOp(ebc47872/rbd_data.4cdec8773331af.0000000000000284/head//6, version: 101770'10045210, data_included: [0~4194304], d ata_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ebc47872/rbd_data.4cdec8773331af .0000000000000284/head//6@101770'10045210, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recov ered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(cf7b872/rbd_data.4d091f7304c844.0000000000010c9d/head//6, version: 101776' 10045226, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecove ryInfo(cf7b872/rbd_data.4d091f7304c844.0000000000010c9d/head//6@101776'10045226, copy_subset: [0~4194304], clone_subset: {}), after_progress: Obj ectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecove ryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -2958> 2014-10-24 11:15:26.097886 7f86b5d2c700 5 -- op tracker -- , seq: 18987, time: 2014-10-24 11:15:26.097885, event: waiting_for_osdmap, re quest: MOSDPGPush(6.872 101783 [PushOp(ebc47872/rbd_data.4cdec8773331af.0000000000000284/head//6, version: 101770'10045210, data_included: [0~419 4304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(ebc47872/rbd_data.4cdec8 773331af.0000000000000284/head//6@101770'10045210, copy_subset: [0~4194304], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, da ta_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recover ed_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),PushOp(cf7b872/rbd_data.4d091f7304c844.0000000000010c9d/head//6, version: 101776'10045226, data_included: [0~4194304], data_size: 4194304, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: Obje ctRecoveryInfo(cf7b872/rbd_data.4d091f7304c844.0000000000010c9d/head//6@101776'10045226, copy_subset: [0~4194304], clone_subset: {}), after_progr ess: ObjectRecoveryProgress(!first, data_recovered_to:4194304, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: Obje ctRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -30> 2014-10-24 11:15:27.761083 7f86a650d700 1 -- 192.168.1.2:6841/28594 --> 192.168.1.4:6844/20497 -- MRecoveryReserve RELEASE pgid: 6.9d8, q uery_epoch: 101783 v2 -- ?+0 0x248df8c0 con 0x206c42c0 -29> 2014-10-24 11:15:27.761151 7f86a650d700 1 -- 192.168.1.2:6841/28594 --> 192.168.1.7:6846/27019 -- MRecoveryReserve RELEASE pgid: 6.9d8, q uery_epoch: 101783 v2 -- ?+0 0x242c0540 con 0x2080f2c0 -28> 2014-10-24 11:15:27.761178 7f86a650d700 5 osd.21 pg_epoch: 101783 pg[6.9d8( v 101783'7781866 (99845'7773549,101783'7781866] local-les=10 1780 n=4310 ec=164 les/c 101780/101775 101779/101779/101779) [21,93,49] r=0 lpr=101779 pi=101771-101778/2 crt=101778'7781858 mlcod 101783'7781865 active snaptrimq=[181a~1,1907~1,19ef~2,19f2~1,19f4~3]] exit Started/Primary/Active/Recovering 1.301663 1 0.000135 -27> 2014-10-24 11:15:27.761288 7f86a650d700 5 osd.21 pg_epoch: 101783 pg[6.9d8( v 101783'7781866 (99845'7773549,101783'7781866] local-les=10 1780 n=4310 ec=164 les/c 101780/101775 101779/101779/101779) [21,93,49] r=0 lpr=101779 pi=101771-101778/2 crt=101778'7781858 mlcod 101783'7781865 active snaptrimq=[181a~1,1907~1,19ef~2,19f2~1,19f4~3]] enter Started/Primary/Active/Recovered -26> 2014-10-24 11:15:27.761381 7f86a650d700 5 osd.21 pg_epoch: 101783 pg[6.9d8( v 101783'7781866 (99845'7773549,101783'7781866] local-les=10 1780 n=4310 ec=164 les/c 101780/101775 101779/101779/101779) [21,93,49] r=0 lpr=101779 pi=101771-101778/2 crt=101778'7781858 mlcod 101783'7781865 active snaptrimq=[181a~1,1907~1,19ef~2,19f2~1,19f4~3]] exit Started/Primary/Active/Recovered 0.000094 0 0.000000 -25> 2014-10-24 11:15:27.761462 7f86a650d700 5 osd.21 pg_epoch: 101783 pg[6.9d8( v 101783'7781866 (99845'7773549,101783'7781866] local-les=10 1780 n=4310 ec=164 les/c 101780/101775 101779/101779/101779) [21,93,49] r=0 lpr=101779 pi=101771-101778/2 crt=101778'7781858 mlcod 101783'7781865 active snaptrimq=[181a~1,1907~1,19ef~2,19f2~1,19f4~3]] enter Started/Primary/Active/Clean -24> 2014-10-24 11:15:27.761524 7f86a6d0e700 5 osd.21 pg_epoch: 101783 pg[6.57c( v 101783'14495519 lc 101770'14494258 (101702'14491206,101783 '14495519] local-les=101780 n=8679 ec=164 les/c 101780/101775 101779/101779/101779) [21,75,110] r=0 lpr=101779 pi=101764-101778/4 crt=101778'1449 5466 mlcod 101770'14494258 active+recovery_wait m=18] exit Started/Primary/Active/WaitLocalRecoveryReserved 238.107976 4 0.000264 -23> 2014-10-24 11:15:27.761702 7f86a650d700 1 -- 192.168.1.2:6841/28594 --> osd.49 192.168.1.4:6844/20497 -- pg_info(1 pgs e101783:6.9d8) v4 - - ?+0 0x213c1000 ecoveryReserve GRANT pgid: 6.57c, query_epoch: 101783 v2 [80/9396] -16> 2014-10-24 11:15:27.764037 7f86b5d2c700 5 -- op tracker -- , seq: 19176, time: 2014-10-24 11:15:27.763860, event: all_read, request: MRe coveryReserve GRANT pgid: 6.57c, query_epoch: 101783 v2 -15> 2014-10-24 11:15:27.764046 7f86b5d2c700 5 -- op tracker -- , seq: 19176, time: 2014-10-24 11:15:27.763978, event: dispatched, request: M RecoveryReserve GRANT pgid: 6.57c, query_epoch: 101783 v2 -14> 2014-10-24 11:15:27.764059 7f86b5d2c700 5 -- op tracker -- , seq: 19176, time: 2014-10-24 11:15:27.764058, event: waiting_for_osdmap, re quest: MRecoveryReserve GRANT pgid: 6.57c, query_epoch: 101783 v2 -13> 2014-10-24 11:15:27.764130 7f86b5d2c700 5 -- op tracker -- , seq: 19176, time: 2014-10-24 11:15:27.764130, event: done, request: MRecove ryReserve GRANT pgid: 6.57c, query_epoch: 101783 v2 -12> 2014-10-24 11:15:27.764173 7f86a4d0a700 1 -- 192.168.1.2:6841/28594 --> 192.168.1.8:6851/5131 -- MRecoveryReserve REQUEST pgid: 6.57c, qu ery_epoch: 101783 v2 -- ?+0 0x21c1e380 con 0x206c4b00 -11> 2014-10-24 11:15:27.765647 7f86b5d2c700 1 -- 192.168.1.2:6841/28594 <== osd.110 192.168.1.8:6851/5131 399 ==== MRecoveryReserve GRANT pgi d: 6.57c, query_epoch: 101783 v2 ==== 26+0+0 (1223105678 0 0) 0x21c1e380 con 0x206c4b00 -10> 2014-10-24 11:15:27.765705 7f86b5d2c700 5 -- op tracker -- , seq: 19177, time: 2014-10-24 11:15:27.765426, event: header_read, request: MRecoveryReserve GRANT pgid: 6.57c, query_epoch: 101783 v2 -9> 2014-10-24 11:15:27.765718 7f86b5d2c700 5 -- op tracker -- , seq: 19177, time: 2014-10-24 11:15:27.765429, event: throttled, request: MR ecoveryReserve GRANT pgid: 6.57c, query_epoch: 101783 v2 -8> 2014-10-24 11:15:27.765725 7f86b5d2c700 5 -- op tracker -- , seq: 19177, time: 2014-10-24 11:15:27.765561, event: all_read, request: MRe coveryReserve GRANT pgid: 6.57c, query_epoch: 101783 v2 -7> 2014-10-24 11:15:27.765733 7f86b5d2c700 5 -- op tracker -- , seq: 19177, time: 2014-10-24 11:15:27.765701, event: dispatched, request: M RecoveryReserve GRANT pgid: 6.57c, query_epoch: 101783 v2 -6> 2014-10-24 11:15:27.765756 7f86b5d2c700 5 -- op tracker -- , seq: 19177, time: 2014-10-24 11:15:27.765755, event: waiting_for_osdmap, re quest: MRecoveryReserve GRANT pgid: 6.57c, query_epoch: 101783 v2 -5> 2014-10-24 11:15:27.765786 7f86b5d2c700 5 -- op tracker -- , seq: 19177, time: 2014-10-24 11:15:27.765786, event: done, request: MRecove ryReserve GRANT pgid: 6.57c, query_epoch: 101783 v2 -4> 2014-10-24 11:15:27.765830 7f86ad51b700 5 osd.21 pg_epoch: 101783 pg[6.57c( v 101783'14495519 lc 101770'14494258 (101702'14491206,101783 '14495519] local-les=101780 n=8679 ec=164 les/c 101780/101775 101779/101779/101779) [21,75,110] r=0 lpr=101779 pi=101764-101778/4 crt=101778'1449 5466 mlcod 101770'14494258 active+recovery_wait m=18] exit Started/Primary/Active/WaitRemoteRecoveryReserved 0.004140 2 0.000369 -3> 2014-10-24 11:15:27.765979 7f86ad51b700 5 osd.21 pg_epoch: 101783 pg[6.57c( v 101783'14495519 lc 101770'14494258 (101702'14491206,101783 '14495519] local-les=101780 n=8679 ec=164 les/c 101780/101775 101779/101779/101779) [21,75,110] r=0 lpr=101779 pi=101764-101778/4 crt=101778'1449 5466 mlcod 101770'14494258 active+recovery_wait m=18] enter Started/Primary/Active/Recovering -2> 2014-10-24 11:15:27.766434 7f86a4509700 1 -- 192.168.1.2:6841/28594 --> 192.168.1.6:6831/3602 -- MOSDPGPull(6.57c 101783 [PullOp(f407257c/ rbd_data.74c0e75ec76d3e.0000000000000321/head//6, recovery_info: ObjectRecoveryInfo(f407257c/rbd_data.74c0e75ec76d3e.0000000000000321/head//6@101 770'14494259, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, dat a_complete:false, omap_recovered_to:, omap_complete:false)),PullOp(13198d7c/rbd_data.4d091f7304c844.0000000000000106/head//6, recovery_info: Obje ctRecoveryInfo(13198d7c/rbd_data.4d091f7304c844.0000000000000106/head//6@101770'14494388, copy_subset: [0~18446744073709551615], clone_subset: 49 ), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)),[44/9396] 35657c/rbd_data.4fd1a75007e195.0000000000000227/head//6, recovery_info: ObjectRecoveryInfo(4935657c/rbd_data.4fd1a75007e195.0000000000000227/head //6@101776'14494494, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to :0, data_complete:false, omap_recovered_to:, omap_complete:false)),PullOp(92aad7c/rbd_data.4d091f7304c844.0000000000011112/head//6, recovery_info : ObjectRecoveryInfo(92aad7c/rbd_data.4d091f7304c844.0000000000011112/head//6@101776'14494550, copy_subset: [0~18446744073709551615], clone_subse t: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x10f1b400 con 0x20696000 -1> 2014-10-24 11:15:27.766541 7f86a4509700 1 -- 192.168.1.2:6841/28594 --> 192.168.1.8:6851/5131 -- MOSDPGPull(6.57c 101783 [PullOp(e0ac457c/ rbd_data.4d091f7304c844.0000000000010a40/head//6, recovery_info: ObjectRecoveryInfo(e0ac457c/rbd_data.4d091f7304c844.0000000000010a40/head//6@101 776'14494552, copy_subset: [0~18446744073709551615], clone_subset: {}), recovery_progress: ObjectRecoveryProgress(first, data_recovered_to:0, dat a_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x20659a00 con 0x206c4b00 0> 2014-10-24 11:15:27.767594 7f86a2505700 -1 *** Caught signal (Segmentation fault) ** in thread 7f86a2505700 ceph version 0.80.7 (6c0127fcb58008793d3c8b62d925bc91963672a3) 1: /usr/bin/ceph-osd() [0x9c830a] 2: (()+0xfcb0) [0x7f86c6009cb0] 3: (ReplicatedPG::trim_object(hobject_t const&)+0x395) [0x8079e5] 4: (ReplicatedPG::TrimmingObjects::react(ReplicatedPG::SnapTrim const&)+0x44c) [0x82215c] 5: (boost::statechart::simple_state<ReplicatedPG::TrimmingObjects, ReplicatedPG::SnapTrimmer, boost::mpl::list<mpl_::na, mpl_::na, mpl_::na, mpl _::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na , mpl_::na, mpl_::na>, (boost::statechart::history_mode)0>::react_impl(boost::statechart::event_base const&, void const*)+0xc0) [0x867390] 6: (boost::statechart::state_machine<ReplicatedPG::SnapTrimmer, ReplicatedPG::NotTrimming, std::allocator<void>, boost::statechart::null_excepti on_translator>::process_queued_events()+0xfb) [0x84d70b] 7: (boost::statechart::state_machine<ReplicatedPG::SnapTrimmer, ReplicatedPG::NotTrimming, std::allocator<void>, boost::statechart::null_excepti on_translator>::process_event(boost::statechart::event_base const&)+0x1e) [0x84d8de] 8: (ReplicatedPG::snap_trimmer()+0x588) [0x7cc118] 9: (OSD::SnapTrimWQ::_process(PG*)+0x14) [0x675f14] 10: (ThreadPool::worker(ThreadPool::WorkThread*)+0x4e6) [0xa9a366] 11: (ThreadPool::WorkThread::entry()+0x10) [0xa9c380] 12: (()+0x7e9a) [0x7f86c6001e9a] 13: (clone()+0x6d) [0x7f86c4efb31d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator [0/9396] 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 0/ 1 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 0/ 5 osd 0/ 5 optracker 0/ 5 objclass 1/ 3 filestore 1/ 3 keyvaluestore 1/ 3 journal 0/ 5 ms 1/ 5 mon 0/10 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.21.log --- end dump of recent events ---
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com