Re: [ceph-users] HEALTH_WARN after upgrade to cuttlefish
According to osdmap e504: 4 osds: 2 up, 2 in you have 2 of 4 osds that are down and out. That may be the issue. David Zafman Senior Developer http://www.inktank.com On May 8, 2013, at 12:05 AM, James Harper james.har...@bendigoit.com.au wrote: I've just upgraded my ceph install to cuttlefish (was 0.60) from Debian. My mon's don't regularly die anymore, or at least haven't so far, but health is always HEALTH_WARN even though I can't see any indication of why: # ceph status health HEALTH_WARN monmap e1: 3 mons at {4=192.168.200.197:6789/0,7=192.168.200.190:6789/0,8=192.168.200.191:6789/0}, election epoch 1104, quorum 0,1,2 4,7,8 osdmap e504: 4 osds: 2 up, 2 in pgmap v210142: 832 pgs: 832 active+clean; 318 GB data, 638 GB used, 1223 GB / 1862 GB avail; 4970B/s rd, 7456B/s wr, 2op/s mdsmap e577: 1/1/1 up {0=7=up:active} Anyone have any idea what might be wrong, or where I can look to find out more? Thanks James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] HEALTH_WARN after upgrade to cuttlefish
On 05/08/2013 08:44 AM, David Zafman wrote: According to osdmap e504: 4 osds: 2 up, 2 in you have 2 of 4 osds that are down and out. That may be the issue. Also, running 'ceph health detail' will give you specifics on what is causing the HEALTH_WARN. -Joao David Zafman Senior Developer http://www.inktank.com On May 8, 2013, at 12:05 AM, James Harper james.har...@bendigoit.com.au wrote: I've just upgraded my ceph install to cuttlefish (was 0.60) from Debian. My mon's don't regularly die anymore, or at least haven't so far, but health is always HEALTH_WARN even though I can't see any indication of why: # ceph status health HEALTH_WARN monmap e1: 3 mons at {4=192.168.200.197:6789/0,7=192.168.200.190:6789/0,8=192.168.200.191:6789/0}, election epoch 1104, quorum 0,1,2 4,7,8 osdmap e504: 4 osds: 2 up, 2 in pgmap v210142: 832 pgs: 832 active+clean; 318 GB data, 638 GB used, 1223 GB / 1862 GB avail; 4970B/s rd, 7456B/s wr, 2op/s mdsmap e577: 1/1/1 up {0=7=up:active} Anyone have any idea what might be wrong, or where I can look to find out more? Thanks James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Joao Eduardo Luis Software Engineer | http://inktank.com | http://ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] HEALTH_WARN after upgrade to cuttlefish
On 05/08/2013 08:44 AM, David Zafman wrote: According to osdmap e504: 4 osds: 2 up, 2 in you have 2 of 4 osds that are down and out. That may be the issue. Also, running 'ceph health detail' will give you specifics on what is causing the HEALTH_WARN. # ceph health detail HEALTH_WARN mon.4 addr 192.168.200.197:6789/0 has 26% avail disk space -- low disk space! I guess that's the problem. Thanks James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] HEALTH_WARN after upgrade to cuttlefish
James, The output says, monmap e1: 3 mons at {4= 192.168.200.197:6789/0,7=192.168.200.190:6789/0,8=192.168.200.191:6789/0}, election epoch 1104, quorum 0,1,2 4,7,8 It looks like you have six OSDs (0,1,2,4,7,8) with only 3 OSDs running. The cluster needs a majority. So you'd need 4 of 6 monitors running. On Wed, May 8, 2013 at 4:32 AM, James Harper james.har...@bendigoit.com.auwrote: On 05/08/2013 08:44 AM, David Zafman wrote: According to osdmap e504: 4 osds: 2 up, 2 in you have 2 of 4 osds that are down and out. That may be the issue. Also, running 'ceph health detail' will give you specifics on what is causing the HEALTH_WARN. # ceph health detail HEALTH_WARN mon.4 addr 192.168.200.197:6789/0 has 26% avail disk space -- low disk space! I guess that's the problem. Thanks James ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- John Wilkins Senior Technical Writer Intank john.wilk...@inktank.com (415) 425-9599 http://inktank.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] HEALTH_WARN after upgrade to cuttlefish
On 05/08/2013 10:57 PM, John Wilkins wrote: James, The output says, monmap e1: 3 mons at {4=192.168.200.197:6789/0,7=192.168.200.190:6789/0,8=192.168.200.191:6789/0 http://192.168.200.197:6789/0,7=192.168.200.190:6789/0,8=192.168.200.191:6789/0}, election epoch 1104, quorum 0,1,2 4,7,8 It looks like you have six OSDs (0,1,2,4,7,8) with only 3 OSDs running. The cluster needs a majority. So you'd need 4 of 6 monitors running. s/OSD/Monitor/ :-) On Wed, May 8, 2013 at 4:32 AM, James Harper james.har...@bendigoit.com.au mailto:james.har...@bendigoit.com.au wrote: On 05/08/2013 08:44 AM, David Zafman wrote: According to osdmap e504: 4 osds: 2 up, 2 in you have 2 of 4 osds that are down and out. That may be the issue. Also, running 'ceph health detail' will give you specifics on what is causing the HEALTH_WARN. # ceph health detail HEALTH_WARN mon.4 addr 192.168.200.197:6789/0 http://192.168.200.197:6789/0 has 26% avail disk space -- low disk space! I guess that's the problem. Thanks James ___ ceph-users mailing list ceph-users@lists.ceph.com mailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- John Wilkins Senior Technical Writer Intank john.wilk...@inktank.com mailto:john.wilk...@inktank.com (415) 425-9599 http://inktank.com -- Joao Eduardo Luis Software Engineer | http://inktank.com | http://ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com