Perhaps I’m not understanding something.. The “extra_probe_peers” ARE the other working monitors in quorum out of the mon_host line in ceph.conf.
In the example below 10.20.1.8 = b20s08; 10.20.10.251 = smon01s; 10.20.10.252 = smon02s The monitor is not reaching out to the other IPs and syncing. I’m able to ping all IPs in the extra_probe_peers list. # ceph --cluster=ceph --admin-daemon /var/run/ceph/ceph-mon.smg01.asok mon_status { "name": "smg01", "rank": 0, "state": "probing", "election_epoch": 0, "quorum": [], "outside_quorum": [ "smg01" ], "extra_probe_peers": [ "10.20.1.8:6789\/0", "10.20.10.251:6789\/0", "10.20.10.252:6789\/0" ], "sync_provider": [], "monmap": { "epoch": 0, "fsid": "693834c1-1f95-4237-ab97-a767b0c0e6e7", "modified": "0.000000", "created": "0.000000", "mons": [ { "rank": 0, "name": "smg01", "addr": "10.20.10.250:6789\/0" }, { "rank": 1, "name": "smon01s", "addr": "0.0.0.0:0\/1" }, { "rank": 2, "name": "smon02s", "addr": "0.0.0.0:0\/2" }, { "rank": 3, "name": "b02s08", "addr": "0.0.0.0:0\/3" } ] } } > On Dec 13, 2015, at 19:18 , Joao Eduardo Luis <j...@suse.de> wrote: > > On 12/13/2015 12:26 PM, deeepdish wrote: >>> >>> This appears to be consistent with a wrongly populated 'mon_host' and >>> 'mon_initial_members' in your ceph.conf. >>> >>> -Joao >> >> >> Thanks Joao. I had a look but my other 3 monitors are working just >> fine. To be clear, I’ve confirmed the same behaviour on other monitor >> nodes that have been removed from the cluster and rebuild with a new IP >> (however same name). > > I'm not entirely sure what you mean, but let me clarify what I meant a bit. > > Existing monitors take their monmap from their own stores. All monitors > in a quorum will see the same monmap. Existing monitors do not care > about the configuration file for their monmap. > > 'mon_host' and 'mon_initial_members' are only used by clients trying to > reach the monitors AND when creating a new monitor. > > Therefore, when creating a new monitor, 'mon_host' must contain the ips > of the existing monitors PLUS the monitor you are creating, and > 'mon_initial_members' must contain the hosts of the existing monitors > PLUS the host of the monitor you are creating. > > Your initial email reflected a lot of other ips on the > 'extra_probe_peers' (which is basically the contents of mon_host during > the probing phase, while the monitor tries to find the other monitors), > which is consistent with mon_host being wrongly populated. > > -Joao
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com