Re: [ceph-users] Manual Installation steps without ceph-deploy
I've seen this before too. CentOS starts up without networking on by default. In my case, the problem was that the monitors cannot form a quorum and OSDs cannot find each other or monitors. Hence, you get that broken pipe error. You either need to have the networking start on startup before the OSDs, or start ceph after you boot up and ensure the network is running properly. The nodes have to be able to reach each other for Ceph to work. As for Ubuntu, I believe the networking is on by default. On Wed, Nov 6, 2013 at 1:35 PM, Trivedi, Narendra wrote: > Hi All, > > > > I did a fresh install of Ceph (this might be like 10th or 11th install) on 4 > new VMs (one admin, one MON and two OSDs) built from CentOS 6.4 (x64) .iso , > did a yum update on all of them. They are all running on vmware ESXi 5.1.0. > I did everything sage et al suggested (i.e. creation of /ceph/osd* and > making sure /etc/ceph is present on all nodes. /etc/ceph gets created all > the ceph-deploy install and contains rbdmap FYI). Unusually, I ended up with > the same problem while activating OSDs (the last 4 lines keep going on and > on forever): > > > > 2013-11-06 14:37:39,626 [ceph_deploy.cli][INFO ] Invoked (1.3): > /usr/bin/ceph-deploy osd activate ceph-node2-osd0-centos-6-4:/ceph/osd0 > ceph-node3-osd1-centos-6-4:/ceph/osd1 > > 2013-11-06 14:37:39,627 [ceph_deploy.osd][DEBUG ] Activating cluster ceph > disks ceph-node2-osd0-centos-6-4:/ceph/osd0: > ceph-node3-osd1-centos-6-4:/ceph/osd1: > > 2013-11-06 14:37:39,901 [ceph-node2-osd0-centos-6-4][DEBUG ] connected to > host: ceph-node2-osd0-centos-6-4 > > 2013-11-06 14:37:39,902 [ceph-node2-osd0-centos-6-4][DEBUG ] detect platform > information from remote host > > 2013-11-06 14:37:39,917 [ceph-node2-osd0-centos-6-4][DEBUG ] detect machine > type > > 2013-11-06 14:37:39,925 [ceph_deploy.osd][INFO ] Distro info: CentOS 6.4 > Final > > 2013-11-06 14:37:39,925 [ceph_deploy.osd][DEBUG ] activating host > ceph-node2-osd0-centos-6-4 disk /ceph/osd0 > > 2013-11-06 14:37:39,925 [ceph_deploy.osd][DEBUG ] will use init type: > sysvinit > > 2013-11-06 14:37:39,925 [ceph-node2-osd0-centos-6-4][INFO ] Running > command: sudo ceph-disk-activate --mark-init sysvinit --mount /ceph/osd0 > > 2013-11-06 14:37:40,145 [ceph-node2-osd0-centos-6-4][ERROR ] 2013-11-06 > 14:37:41.075310 7fac2414c700 0 -- :/1029546 >> 10.12.0.70:6789/0 > pipe(0x7fac20024480 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7fac200246e0).fault > > 2013-11-06 14:37:43,167 [ceph-node2-osd0-centos-6-4][ERROR ] 2013-11-06 > 14:37:44.071697 7fac1ebfd700 0 -- :/1029546 >> 10.12.0.70:6789/0 > pipe(0x7fac14000c00 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7fac14000e60).fault > > 2013-11-06 14:37:46,140 [ceph-node2-osd0-centos-6-4][ERROR ] 2013-11-06 > 14:37:47.071938 7fac2414c700 0 -- :/1029546 >> 10.12.0.70:6789/0 > pipe(0x7fac14003010 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7fac14003270).fault > > 2013-11-06 14:37:50,165 [ceph-node2-osd0-centos-6-4][ERROR ] 2013-11-06 > 14:37:51.071245 7fac1ebfd700 0 -- :/1029546 >> 10.12.0.70:6789/0 > pipe(0x7fac14003a70 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7fac14003cd0).fault > > > > It might be bad luck but I want to try a manual installation without > ceph-deploy because it seems I am jinxed with ceph-deploy. Could anyone > please forward me the steps. I am happy to share the ceph.log with anyone > who would like to research on this error but I don’t a have clue. > > > > > > Thanks a lot! > > Narendra Trivedi | savviscloud > > > > > This message contains information which may be confidential and/or > privileged. Unless you are the intended recipient (or authorized to receive > for the intended recipient), you may not read, use, copy or disclose to > anyone the message or any information contained in the message. If you have > received the message in error, please advise the sender by reply e-mail and > delete the message and any attachment(s) thereto without retaining any > copies. > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- John Wilkins Senior Technical Writer Intank john.wilk...@inktank.com (415) 425-9599 http://inktank.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Manual Installation steps without ceph-deploy
Unfortunately, I don't have that luxury. Thanks! Narendra -Original Message- From: ceph-users-boun...@lists.ceph.com [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of ja...@peacon.co.uk Sent: Wednesday, November 06, 2013 4:43 PM To: ceph-users@lists.ceph.com Subject: Re: [ceph-users] Manual Installation steps without ceph-deploy I also had some difficulty with ceph-deploy on CentOS. I eventually moved to Ubuntu 13.04 - and haven't looked back. On 2013-11-06 21:35, Trivedi, Narendra wrote: > Hi All, > > I did a fresh install of Ceph (this might be like 10th or 11th > install) on 4 new VMs (one admin, one MON and two OSDs) built from > CentOS 6.4 (x64)... it seems I am jinxed with ceph-deploy. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com This message contains information which may be confidential and/or privileged. Unless you are the intended recipient (or authorized to receive for the intended recipient), you may not read, use, copy or disclose to anyone the message or any information contained in the message. If you have received the message in error, please advise the sender by reply e-mail and delete the message and any attachment(s) thereto without retaining any copies. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Manual Installation steps without ceph-deploy
I also had some difficulty with ceph-deploy on CentOS. I eventually moved to Ubuntu 13.04 - and haven't looked back. On 2013-11-06 21:35, Trivedi, Narendra wrote: Hi All, I did a fresh install of Ceph (this might be like 10th or 11th install) on 4 new VMs (one admin, one MON and two OSDs) built from CentOS 6.4 (x64)... it seems I am jinxed with ceph-deploy. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Manual Installation steps without ceph-deploy
Hi All, I did a fresh install of Ceph (this might be like 10th or 11th install) on 4 new VMs (one admin, one MON and two OSDs) built from CentOS 6.4 (x64) .iso , did a yum update on all of them. They are all running on vmware ESXi 5.1.0. I did everything sage et al suggested (i.e. creation of /ceph/osd* and making sure /etc/ceph is present on all nodes. /etc/ceph gets created all the ceph-deploy install and contains rbdmap FYI). Unusually, I ended up with the same problem while activating OSDs (the last 4 lines keep going on and on forever): 2013-11-06 14:37:39,626 [ceph_deploy.cli][INFO ] Invoked (1.3): /usr/bin/ceph-deploy osd activate ceph-node2-osd0-centos-6-4:/ceph/osd0 ceph-node3-osd1-centos-6-4:/ceph/osd1 2013-11-06 14:37:39,627 [ceph_deploy.osd][DEBUG ] Activating cluster ceph disks ceph-node2-osd0-centos-6-4:/ceph/osd0: ceph-node3-osd1-centos-6-4:/ceph/osd1: 2013-11-06 14:37:39,901 [ceph-node2-osd0-centos-6-4][DEBUG ] connected to host: ceph-node2-osd0-centos-6-4 2013-11-06 14:37:39,902 [ceph-node2-osd0-centos-6-4][DEBUG ] detect platform information from remote host 2013-11-06 14:37:39,917 [ceph-node2-osd0-centos-6-4][DEBUG ] detect machine type 2013-11-06 14:37:39,925 [ceph_deploy.osd][INFO ] Distro info: CentOS 6.4 Final 2013-11-06 14:37:39,925 [ceph_deploy.osd][DEBUG ] activating host ceph-node2-osd0-centos-6-4 disk /ceph/osd0 2013-11-06 14:37:39,925 [ceph_deploy.osd][DEBUG ] will use init type: sysvinit 2013-11-06 14:37:39,925 [ceph-node2-osd0-centos-6-4][INFO ] Running command: sudo ceph-disk-activate --mark-init sysvinit --mount /ceph/osd0 2013-11-06 14:37:40,145 [ceph-node2-osd0-centos-6-4][ERROR ] 2013-11-06 14:37:41.075310 7fac2414c700 0 -- :/1029546 >> 10.12.0.70:6789/0 pipe(0x7fac20024480 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7fac200246e0).fault 2013-11-06 14:37:43,167 [ceph-node2-osd0-centos-6-4][ERROR ] 2013-11-06 14:37:44.071697 7fac1ebfd700 0 -- :/1029546 >> 10.12.0.70:6789/0 pipe(0x7fac14000c00 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7fac14000e60).fault 2013-11-06 14:37:46,140 [ceph-node2-osd0-centos-6-4][ERROR ] 2013-11-06 14:37:47.071938 7fac2414c700 0 -- :/1029546 >> 10.12.0.70:6789/0 pipe(0x7fac14003010 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7fac14003270).fault 2013-11-06 14:37:50,165 [ceph-node2-osd0-centos-6-4][ERROR ] 2013-11-06 14:37:51.071245 7fac1ebfd700 0 -- :/1029546 >> 10.12.0.70:6789/0 pipe(0x7fac14003a70 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7fac14003cd0).fault It might be bad luck but I want to try a manual installation without ceph-deploy because it seems I am jinxed with ceph-deploy. Could anyone please forward me the steps. I am happy to share the ceph.log with anyone who would like to research on this error but I don't a have clue. Thanks a lot! Narendra Trivedi | savviscloud This message contains information which may be confidential and/or privileged. Unless you are the intended recipient (or authorized to receive for the intended recipient), you may not read, use, copy or disclose to anyone the message or any information contained in the message. If you have received the message in error, please advise the sender by reply e-mail and delete the message and any attachment(s) thereto without retaining any copies.___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
