Some test results:
1.If using custom admin_secret for MONs, can see
/etc/ceph/ceph.client.admin.keyring which was missing in previous attempts.
2.stop firewalld on all MONs.
Both attempts gave same result as before: got stuck at collect admin and
bootstrap keys as before.
Appreciate any clue.
Instead of deploying mimic, should we deploy luminous with latest ansible and
ceph-ansible 4.0 or master, then upgrade to mimic ?
Thanks a lot.
From: ceph-users <ceph-users-boun...@lists.ceph.com> On Behalf Of ST Wong (ITSC)
Sent: Tuesday, May 7, 2019 11:48 AM
To: solarflow99 <solarflo...@gmail.com>
Cc: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] ceph-create-keys loops
Some more information:
Rpm installed on MONS:
python-cephfs-13.2.5-0.el7.x86_64
ceph-base-13.2.5-0.el7.x86_64
libcephfs2-13.2.5-0.el7.x86_64
ceph-common-13.2.5-0.el7.x86_64
ceph-selinux-13.2.5-0.el7.x86_64
ceph-mon-13.2.5-0.el7.x86_64
Doing a mon_status gives following. Only the local host running the command
can see addr/pub_addr correctly while all other MONs have addr/pub_addr
“0.0.0.0”. Besides, the mon feature shows it’s luminous while we’re installing
mimic.
======================
[root@cphmon2a ~]# ceph --admin-daemon /var/run/ceph/ceph-mon.cphmon2a.asok
mon_status
{
"name": "cphmon2a",
"rank": 0,
"state": "probing",
"election_epoch": 0,
"quorum": [],
"features": {
"required_con": "0",
"required_mon": [],
"quorum_con": "0",
"quorum_mon": []
},
"outside_quorum": [
"cphmon2a"
],
"extra_probe_peers": [],
"sync_provider": [],
"monmap": {
"epoch": 0,
"fsid": "cf746667-3132-476d-a287-19143404cf39",
"modified": "0.000000",
"created": "0.000000",
"features": {
"persistent": [],
"optional": []
},
"mons": [
{
"rank": 0,
"name": "cphmon2a",
"addr": "123.123.7.93:6789/0",
"public_addr": "123.123.7.93:6789/0"
},
{
"rank": 1,
"name": "cphmon1a",
"addr": "0.0.0.0:0/1",
"public_addr": "0.0.0.0:0/1"
},
{
"rank": 2,
"name": "cphmon3a",
"addr": "0.0.0.0:0/2",
"public_addr": "0.0.0.0:0/2"
},
{
"rank": 3,
"name": "cphmon4b",
"addr": "0.0.0.0:0/3",
"public_addr": "0.0.0.0:0/3"
},
{
"rank": 4,
"name": "cphmon5b",
"addr": "0.0.0.0:0/4",
"public_addr": "0.0.0.0:0/4"
},
{
"rank": 5,
"name": "cphmon6b",
"addr": "0.0.0.0:0/5",
"public_addr": "0.0.0.0:0/5"
}
]
},
"feature_map": {
"mon": [
{
"features": "0x3ffddff8ffacfffb",
"release": "luminous",
"num": 1
}
]
}
}
======================
Thanks a lot.
From: ceph-users
<ceph-users-boun...@lists.ceph.com<mailto:ceph-users-boun...@lists.ceph.com>>
On Behalf Of ST Wong (ITSC)
Sent: Tuesday, May 7, 2019 7:59 AM
To: solarflow99 <solarflo...@gmail.com<mailto:solarflo...@gmail.com>>
Cc: ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
Subject: Re: [ceph-users] ceph-create-keys loops
yes, we’re using 3.2 stable, on RHEL 7. Thanks.
From: solarflow99 <solarflo...@gmail.com<mailto:solarflo...@gmail.com>>
Sent: Tuesday, May 7, 2019 1:40 AM
To: ST Wong (ITSC) <s...@itsc.cuhk.edu.hk<mailto:s...@itsc.cuhk.edu.hk>>
Cc: ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
Subject: Re: [ceph-users] ceph-create-keys loops
you mention the version of ansible, that is right. How about the branch of
ceph-ansible? should be 3.2-stable, what OS? I haven't come across this
problem myself, a lot of other ones.
On Mon, May 6, 2019 at 3:47 AM ST Wong (ITSC)
<s...@itsc.cuhk.edu.hk<mailto:s...@itsc.cuhk.edu.hk>> wrote:
Hi all,
I’ve problem in deploying mimic using ceph-ansible at following step:
---------- cut here -----------
TASK [ceph-mon : collect admin and bootstrap keys] *****************************
Monday 06 May 2019 17:01:23 +0800 (0:00:00.854) 0:05:38.899 ************
fatal: [cphmon3a]: FAILED! => {"changed": false, "cmd": ["ceph-create-keys",
"--cluster", "ceph", "-i", "cphmon3a", "-t", "600"], "delta": "0:11:24.675833",
"end": "2019-05-06 17:12:48.500996", "msg": "non-zero return code", "rc": 1,
"start": "2019-05-06 17:01:23.825163", "stderr":
"INFO:ceph-create-keys:ceph-mon is not in quorum: u'probing'\n
INFO:ceph-create-keys:ceph-mon is not in quorum:
u'probing'\nINFO:ceph-create-keys:ceph-mon is not in quorum:
u'probing'\nINFO:ceph-create-keys:ceph-mon is not in quorum:
u'probing'\nINFO:ceph-create-keys:ceph-mon is not in quorum:
u'probing'\nINFO:ceph-create-keys:ceph-mon is not in quorum:
u'probing'\nINFO:ceph-create-keys:ceph-mon is not in quorum:
u'probing'\nINFO:ceph-create-keys:ceph-mon is not in quorum:
u'probing'\nINFO:ceph-create-keys:ceph-mon is not in quorum: u'probing'\n
---------- cut here -----------
There are 2 NIC on all MONs. The site.yml contains some basic configuration:
----------- site.yml --------
dummy:
ceph_origin: repository
ceph_repository: community
ceph_stable_release: mimic
public_network: "123.123.7.0/24<http://123.123.7.0/24>"
<-- fake ip range of our public network
cluster_network: "192.168.77.0/24<http://192.168.77.0/24>"
monitor_interface: p2p1
----------- site.yml --------
And the ceph.conf created on MONs :
-------- ceph.conf -------
# Please do not change this file directly since it is managed by Ansible and
will be overwritten
[global]
fsid = 17db45c6-b5ac-47e8-b5cb-b3e5215f4af4
mon initial members = cphmon1a,cphmon2a,cphmon3a,cphmon4b,cphmon5b,cphmon6b
osd pool default crush rule = -1
mon host =
123.123.7.92,123.123.7.93,123.123.7.94,123.123.7.95,123.123.7.96,123.123.7.97
public network = 123.123.7.0/24<http://123.123.7.0/24>
cluster network = 192.168.77.0/24<http://192.168.77.0/24>
-------- ceph.conf -------
Also get same error if running following on one of the MONs:
#ceph-create-keys --cluster ceph --id cphmon1a
INFO:ceph-create-keys:ceph-mon is not in quorum: u'probing'
INFO:ceph-create-keys:ceph-mon is not in quorum: u'probing'
INFO:ceph-create-keys:ceph-mon is not in quorum: u'probing'
INFO:ceph-create-keys:ceph-mon is not in quorum: u'probing'
We’re using Ansible 2.6 (for deploying Mimic using ceph-ansible).
Would anyone please help?
Thanks a lot.
/st wong
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
