Re: [ceph-users] infernalis and jewel upgrades...

2016-04-15 Thread hjcho616
Deleted old messages... Here is what I am seeing..  Home contains copy from OSD2. root@OSD1:/var/lib/ceph/osd# find ./ceph-*/current/meta | grep osdmap | grep 16024./ceph-0/current/meta/DIR_E/DIR_3/inc\uosdmap.16024__0_46887E3E__none./ceph-0/current/meta/DIR_9/DIR_D/osdmap.16024__0_4E98A1D9__none

Re: [ceph-users] infernalis and jewel upgrades...

2016-04-15 Thread huang jun
can you find the full osdmap.16024 in other osd? seems like the osd::init doesnt read the incremental osdmap but the full osdmap, if you find it, then copy to osd.3. 2016-04-16 13:27 GMT+08:00 hjcho616 : > I found below file missing on osd.3 so I copied over. Still fails with the > similar messag

Re: [ceph-users] infernalis and jewel upgrades...

2016-04-15 Thread huang jun
yes, it's a incremental osdmap, does the file size is correct? you can compare it with the same file in other osd. If it's not the same, you can overwrite it with the right one. 2016-04-16 13:11 GMT+08:00 hjcho616 : > Is this it? > > root@OSD2:/var/lib/ceph/osd/ceph-3/current/meta# find ./ | grep

Re: [ceph-users] infernalis and jewel upgrades...

2016-04-15 Thread huang jun
First, you should check whether file osdmap.16024 exists in your osd.3/current/meta dir, if not, you can copy it from other OSD who has it. 2016-04-16 12:36 GMT+08:00 hjcho616 : > Here is what I get wtih debug_osd = 20. > > 2016-04-15 23:28:24.429063 7f9ca0a5b800 0 set uid:gid to 1001:1001 > (ce

Re: [ceph-users] infernalis and jewel upgrades...

2016-04-15 Thread huang jun
Can you set 'debug_osd = 20' in ceph.conf and restart the osd again, and post the corrupt log. I doubt it's problem related to "0 byte osdmap" decode problem? 2016-04-16 12:14 GMT+08:00 hjcho616 : > I've been successfully running cephfs on my Debian Jessies for a while and > one day after power ou

[ceph-users] infernalis and jewel upgrades...

2016-04-15 Thread hjcho616
I've been successfully running cephfs on my Debian Jessies for a while and one day after power outage, MDS wasn't happy.  MDS crashing after it was done loading, increasing the memory utilization quite a bit.  I was running infernalis 9.2.0 and did successful upgrade from Hammer before... so I t