Just curious why it wouldn't work as long as the IPs were reachable? Is there something going on in layer 2 with Ceph that wouldn't survive a trip across a router?
On Wed, Aug 30, 2017 at 1:52 PM, David Turner <drakonst...@gmail.com> wrote: > ALL OSDs need to be running the same private network at the same time. > ALL clients, RGW, OSD, MON, MGR, MDS, etc, etc need to be running on the > same public network at the same time. You cannot do this as a one at a > time migration to the new IP space. Even if all of the servers can still > communicate via routing, it just won't work. Changing the public/private > network addresses for a cluster requires full cluster down time. > > On Wed, Aug 30, 2017 at 11:09 AM Ben Morrice <ben.morr...@epfl.ch> wrote: > >> Hello >> >> We have a small cluster that we need to move to a different network in >> the same datacentre. >> >> My workflow was the following (for a single OSD host), but I failed >> (further details below) >> >> 1) ceph osd set noout >> 2) stop ceph-osd processes >> 3) change IP, gateway, domain (short hostname is the same), VLAN >> 4) change references of OLD IP (cluster and public network) in >> /etc/ceph/ceph.conf with NEW IP (see [1]) >> 5) start a single OSD process >> >> This seems to work as the NEW IP can communicate with mon hosts and osd >> hosts on the OLD network, the OSD is booted and is visible via 'ceph -w' >> however after a few seconds the OSD drops with messages such as the >> below in it's log file >> >> heartbeat_check: no reply from 10.1.1.100:6818 osd.14 ever on either >> front or back, first ping sent 2017-08-30 16:42:14.692210 (cutoff >> 2017-08-30 16:42:24.962245) >> >> There are logs like the above for every OSD server/process >> >> and then eventually a >> >> 2017-08-30 16:42:14.486275 7f6d2c966700 0 log_channel(cluster) log >> [WRN] : map e85351 wrongly marked me down >> >> >> Am I missing something obvious to reconfigure the network on a OSD host? >> >> >> >> [1] >> >> OLD >> [osd.0] >> host = sn01 >> devs = /dev/sdi >> cluster addr = 10.1.1.101 >> public addr = 10.1.1.101 >> NEW >> [osd.0] >> host = sn01 >> devs = /dev/sdi >> cluster addr = 10.1.2.101 >> public addr = 10.1.2.101 >> >> -- >> Kind regards, >> >> Ben Morrice >> >> ______________________________________________________________________ >> Ben Morrice | e: ben.morr...@epfl.ch | t: +41-21-693-9670 >> <+41%2021%20693%2096%2070> >> EPFL / BBP >> Biotech Campus >> Chemin des Mines 9 >> 1202 Geneva >> Switzerland >> >> _______________________________________________ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
