On Thu, Dec 5, 2013 at 7:12 AM, Karan Singh <ksi...@csc.fi> wrote: > what ceph status and ceph mon_status output says , did you checked logs > after this anything interesting there ? > > Many Thanks > Karan Singh > > > ----- Original Message ----- > From: "Jan Kalcic" <jan...@gmail.com> > To: "Karan Singh" <ksi...@csc.fi> > Cc: ceph-users@lists.ceph.com > Sent: Thursday, 5 December, 2013 12:58:33 PM > Subject: Re: [ceph-users] IO error: lock > /var/lib/ceph/mon/ceph-ceph-node1/store.db/LOCK: Resource temporarily > unavailable > > Hi Karan, > > On 12/05/2013 10:31 AM, Karan Singh wrote: >> Hello Jan >> >> I faced similar kind of errors and these are really annoying. I tried this >> and worked for me. > > Glad to know I am not alone :-) , though this sounds like a not really > robust procedure... >> >> 1. Your ceph-node1 is now a monitor node, but it cannot form quorum. >> 2. check monitor logs from ceph-node1 on /var/lib/ceph directory. This >> will give you more strength. >> 3. You might need to manually add monitor in cluster. >> >> >> Below is my solution ( you should change it as per your hostname and IP >> etc) >> >> >> >> #### login on monitor node and execute commands like below >> ceph mon add ceph-mon2 192.168.1.33:6789 >> ceph-mon -i ceph-mon2 --public-addr 192.168.1.33:6789 >> service ceph status >> service ceph restart >> ps -ef | grep ceph >> >> In case ceph service is not showing as running in service ceph status >> command , however ps -ef | grep ceph , shows monitor running , kill the >> process manually and restart ceph services , monitor now should start working >> >> # service ceph status >> # ps -ef | grep ceph >> # kill -9 6554 >> > > I have played around with your suggestions but I could not solve.. I > removed the existing directory in /var/lib/ceph/mon for the mon node and > it seems I could go a step ahead: > > > ceph-admin # ceph-deploy mon create ceph-node1 > [ceph_deploy.cli][INFO ] Invoked (1.3.3): /usr/bin/ceph-deploy mon > create ceph-node1 > [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph-node1 > [ceph_deploy.mon][DEBUG ] detecting platform for host ceph-node1 ... > [ceph-node1][DEBUG ] connected to host: ceph-node1 > [ceph-node1][DEBUG ] detect platform information from remote host > [ceph-node1][DEBUG ] detect machine type > [ceph_deploy.mon][INFO ] distro info: SUSE Linux Enterprise Server 11 > x86_64 > [ceph-node1][DEBUG ] determining if provided host has same hostname in > remote > [ceph-node1][DEBUG ] get remote short hostname > [ceph-node1][DEBUG ] deploying mon to ceph-node1 > [ceph-node1][DEBUG ] get remote short hostname > [ceph-node1][DEBUG ] remote hostname: ceph-node1 > [ceph-node1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf > [ceph-node1][DEBUG ] create the mon path if it does not exist > [ceph-node1][DEBUG ] checking for done path: > /var/lib/ceph/mon/ceph-ceph-node1/done > [ceph-node1][DEBUG ] done path does not exist: > /var/lib/ceph/mon/ceph-ceph-node1/done > [ceph-node1][INFO ] creating keyring file: > /var/lib/ceph/tmp/ceph-ceph-node1.mon.keyring > [ceph-node1][DEBUG ] create the monitor keyring file > [ceph-node1][INFO ] Running command: sudo ceph-mon --cluster ceph > --mkfs -i ceph-node1 --keyring /var/lib/ceph/tmp/ceph-ceph-node1.mon.keyring > [ceph-node1][DEBUG ] ceph-mon: mon.noname-a 172.17.43.11:6789/0 is > local, renaming to mon.ceph-node1 > [ceph-node1][DEBUG ] ceph-mon: set fsid to > 18203301-f70b-40f1-b93e-fde4731f3bee > [ceph-node1][DEBUG ] ceph-mon: created monfs at > /var/lib/ceph/mon/ceph-ceph-node1 for mon.ceph-node1 > [ceph-node1][INFO ] unlinking keyring file > /var/lib/ceph/tmp/ceph-ceph-node1.mon.keyring > [ceph-node1][DEBUG ] create a done file to avoid re-doing the mon deployment > [ceph-node1][DEBUG ] create the init path if it does not exist > [ceph-node1][DEBUG ] locating the `service` executable... > [ceph-node1][INFO ] Running command: sudo rcceph -c /etc/ceph/ceph.conf > start mon.ceph-node1 > [ceph-node1][DEBUG ] === mon.ceph-node1 === > [ceph-node1][DEBUG ] Starting Ceph mon.ceph-node1 on ceph-node1... > [ceph-node1][DEBUG ] Starting ceph-create-keys on ceph-node1... > [ceph-node1][WARNIN] No data was received after 7 seconds, disconnecting... > [ceph-node1][INFO ] Running command: sudo ceph --cluster=ceph > --admin-daemon /var/run/ceph/ceph-mon.ceph-node1.asok mon_status > > But then the command is pending and nothing happens. If I kill it and > then restart it with /etc/init.d/ceph start it is the same: > > ceph-node1 # /etc/init.d/ceph start > === mon.ceph-node1 === > Starting Ceph mon.ceph-node1 on ceph-node1... > > Anything else?
Do you have iptables enabled? If so, either open the ports for the monitors to be able to talk or disable it and try again > > Thanks, > Jan > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com