I am trying to deploy a cluster with ceph-deploy. I installed ceph 0.72.2 from the rpm repositories. Running "ceph-deploy mon create-initial" creates /var/lib/ceph etc. on all the nodes, but on all nodes I get a warning:
[hvrrzceph2][DEBUG ] Starting Ceph mon.hvrrzceph2 on hvrrzceph2... [hvrrzceph2][DEBUG ] Starting ceph-create-keys on hvrrzceph2... [hvrrzceph2][WARNING] No data was received after 7 seconds, disconnecting... and afterwards, ceph -s cannot connect to the cluster. No client.admin keyring is created in ceph.conf. Then I attempted to create one monitor only on one node. Again, no keys for client.admin, bootstrap-mds and bootstrap-osd were created. Here is the complete log of this attempt: hvrrzceph1:~/my-cluster # ceph-deploy new hvrrzceph1 [ceph_deploy.cli][INFO ] Invoked (1.4.0): /usr/bin/ceph-deploy new hvrrzceph1 [ceph_deploy.new][DEBUG ] Creating new cluster named ceph [ceph_deploy.new][DEBUG ] Resolving host hvrrzceph1 [ceph_deploy.new][DEBUG ] Monitor hvrrzceph1 at 10.1.1.239 [ceph_deploy.new][INFO ] making sure passwordless SSH succeeds [ceph_deploy.new][DEBUG ] Monitor initial members are ['hvrrzceph1'] [ceph_deploy.new][DEBUG ] Monitor addrs are ['10.1.1.239'] [ceph_deploy.new][DEBUG ] Creating a random mon key... [ceph_deploy.new][DEBUG ] Writing initial config to ceph.conf... [ceph_deploy.new][DEBUG ] Writing monitor keyring to ceph.mon.keyring... hvrrzceph1:~/my-cluster # vi ceph.conf [added "cluster network" + "public network"; leaving them away doesnt change anything] hvrrzceph1:~/my-cluster # cat ceph.conf [global] auth_service_required = cephx filestore_xattr_use_omap = true auth_client_required = cephx auth_cluster_required = cephx mon_host = 10.1.1.239 mon_initial_members = hvrrzceph1 fsid = ec73a230-f645-456f-9523-9a03621d18dd cluster_network = 10.112.0.0/16 public_network = 10.111.0.0/16 hvrrzceph1:~/my-cluster # ceph-deploy mon create-initial [ceph_deploy.cli][INFO ] Invoked (1.4.0): /usr/bin/ceph-deploy mon create-initial [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts hvrrzceph1 [ceph_deploy.mon][DEBUG ] detecting platform for host hvrrzceph1 ... [hvrrzceph1][DEBUG ] connected to host: hvrrzceph1 [hvrrzceph1][DEBUG ] detect platform information from remote host [hvrrzceph1][DEBUG ] detect machine type [ceph_deploy.mon][INFO ] distro info: SUSE Linux Enterprise Server 11 x86_64 [hvrrzceph1][DEBUG ] determining if provided host has same hostname in remote [hvrrzceph1][DEBUG ] get remote short hostname [hvrrzceph1][DEBUG ] deploying mon to hvrrzceph1 [hvrrzceph1][DEBUG ] get remote short hostname [hvrrzceph1][DEBUG ] remote hostname: hvrrzceph1 [hvrrzceph1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [hvrrzceph1][DEBUG ] create the mon path if it does not exist [hvrrzceph1][DEBUG ] checking for done path: /var/lib/ceph/mon/ceph-hvrrzceph1/done [hvrrzceph1][DEBUG ] done path does not exist: /var/lib/ceph/mon/ceph-hvrrzceph1/done [hvrrzceph1][INFO ] creating keyring file: /var/lib/ceph/tmp/ceph-hvrrzceph1.mon.keyring [hvrrzceph1][DEBUG ] create the monitor keyring file [hvrrzceph1][INFO ] Running command: ceph-mon --cluster ceph --mkfs -i hvrrzceph1 --keyring /var/lib/ceph/tmp/ceph-hvrrzceph1.mon.keyring [hvrrzceph1][DEBUG ] ceph-mon: set fsid to ec73a230-f645-456f-9523-9a03621d18dd [hvrrzceph1][DEBUG ] ceph-mon: created monfs at /var/lib/ceph/mon/ceph-hvrrzceph1 for mon.hvrrzceph1 [hvrrzceph1][INFO ] unlinking keyring file /var/lib/ceph/tmp/ceph-hvrrzceph1.mon.keyring [hvrrzceph1][DEBUG ] create a done file to avoid re-doing the mon deployment [hvrrzceph1][DEBUG ] create the init path if it does not exist [hvrrzceph1][INFO ] Running command: rcceph -c /etc/ceph/ceph.conf start mon.hvrrzceph1 [hvrrzceph1][DEBUG ] === mon.hvrrzceph1 === [hvrrzceph1][DEBUG ] Starting Ceph mon.hvrrzceph1 on hvrrzceph1... [hvrrzceph1][DEBUG ] Starting ceph-create-keys on hvrrzceph1... [hvrrzceph1][WARNIN] No data was received after 7 seconds, disconnecting... [hvrrzceph1][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.hvrrzceph1.asok mon_status [hvrrzceph1][DEBUG ] ********************************************************************** ********** [hvrrzceph1][DEBUG ] status for monitor: mon.hvrrzceph1 [hvrrzceph1][DEBUG ] { [hvrrzceph1][DEBUG ] "election_epoch": 2, [hvrrzceph1][DEBUG ] "extra_probe_peers": [ [hvrrzceph1][DEBUG ] "10.1.1.239:6789/0" [hvrrzceph1][DEBUG ] ], [hvrrzceph1][DEBUG ] "monmap": { [hvrrzceph1][DEBUG ] "created": "0.000000", [hvrrzceph1][DEBUG ] "epoch": 1, [hvrrzceph1][DEBUG ] "fsid": "ec73a230-f645-456f-9523-9a03621d18dd", [hvrrzceph1][DEBUG ] "modified": "0.000000", [hvrrzceph1][DEBUG ] "mons": [ [hvrrzceph1][DEBUG ] { [hvrrzceph1][DEBUG ] "addr": "10.111.3.1:6789/0", [hvrrzceph1][DEBUG ] "name": "hvrrzceph1", [hvrrzceph1][DEBUG ] "rank": 0 [hvrrzceph1][DEBUG ] } [hvrrzceph1][DEBUG ] ] [hvrrzceph1][DEBUG ] }, [hvrrzceph1][DEBUG ] "name": "hvrrzceph1", [hvrrzceph1][DEBUG ] "outside_quorum": [], [hvrrzceph1][DEBUG ] "quorum": [ [hvrrzceph1][DEBUG ] 0 [hvrrzceph1][DEBUG ] ], [hvrrzceph1][DEBUG ] "rank": 0, [hvrrzceph1][DEBUG ] "state": "leader", [hvrrzceph1][DEBUG ] "sync_provider": [] [hvrrzceph1][DEBUG ] } [hvrrzceph1][DEBUG ] ********************************************************************** ********** [hvrrzceph1][INFO ] monitor: mon.hvrrzceph1 is running [hvrrzceph1][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.hvrrzceph1.asok mon_status [ceph_deploy.mon][INFO ] processing monitor mon.hvrrzceph1 [hvrrzceph1][DEBUG ] connected to host: hvrrzceph1 [hvrrzceph1][INFO ] Running command: ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.hvrrzceph1.asok mon_status [ceph_deploy.mon][INFO ] mon.hvrrzceph1 monitor has reached quorum! [ceph_deploy.mon][INFO ] all initial monitors are running and have formed quorum [ceph_deploy.mon][INFO ] Running gatherkeys... [ceph_deploy.gatherkeys][DEBUG ] Checking hvrrzceph1 for /etc/ceph/ceph.client.admin.keyring [hvrrzceph1][DEBUG ] connected to host: hvrrzceph1 [hvrrzceph1][DEBUG ] detect platform information from remote host [hvrrzceph1][DEBUG ] detect machine type [hvrrzceph1][DEBUG ] fetch remote file [ceph_deploy.gatherkeys][WARNIN] Unable to find /etc/ceph/ceph.client.admin.keyring on ['hvrrzceph1'] [ceph_deploy.gatherkeys][DEBUG ] Have ceph.mon.keyring [ceph_deploy.gatherkeys][DEBUG ] Checking hvrrzceph1 for /var/lib/ceph/bootstrap-osd/ceph.keyring [hvrrzceph1][DEBUG ] connected to host: hvrrzceph1 [hvrrzceph1][DEBUG ] detect platform information from remote host [hvrrzceph1][DEBUG ] detect machine type [hvrrzceph1][DEBUG ] fetch remote file [ceph_deploy.gatherkeys][WARNIN] Unable to find /var/lib/ceph/bootstrap-osd/ceph.keyring on ['hvrrzceph1'] [ceph_deploy.gatherkeys][DEBUG ] Checking hvrrzceph1 for /var/lib/ceph/bootstrap-mds/ceph.keyring [hvrrzceph1][DEBUG ] connected to host: hvrrzceph1 [hvrrzceph1][DEBUG ] detect platform information from remote host [hvrrzceph1][DEBUG ] detect machine type [hvrrzceph1][DEBUG ] fetch remote file [ceph_deploy.gatherkeys][WARNIN] Unable to find /var/lib/ceph/bootstrap-mds/ceph.keyring on ['hvrrzceph1'] hvrrzceph1:~/my-cluster # I had already a running cluster, and ceph-deploy had worked ... :-( Any hints where I can find what is missing? TIA. -- Diedrich Ehlerding, Fujitsu Technology Solutions GmbH, FTS CE SC PS&IS W, Hildesheimer Str 25, D-30880 Laatzen Fon +49 511 8489-1806, Fax -251806, Mobil +49 173 2464758 Firmenangaben: http://de.ts.fujitsu.com/imprint.html _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com