Hi,
Thanks for your help.
I changed IP addresses of OSD nodes successfully. When changing IP address on
MON, it works except that the MON only listens on v2 port 3300 after adding the
MON back to the cluster. Previously the MON listens on both v1 (6789) and v2
(3300).
Besides, can't add both v1 and v2 entries manually using monmaptool like
following. Only the 2nd one will get added.
monmaptool -add node1 v1:10.0.1.1:6789, v2:10.0.1.1:3330 {tmp}/{filename}
the monmap now looks like following:
min_mon_release 14 (nautilus)
0: [v2:10.0.1.92:3300/0,v1:10.0.1.92:6789/0] mon.cmon2
1: [v2:10.0.1.93:3300/0,v1:10.0.1.93:6789/0] mon.cmon2
2: [v2:10.0.1.94:3300/0,v1:10.0.1.94:6789/0] mon.cmon3
3: [v2:10.0.1.95:3300/0,v1:10.0.1.95:6789/0] mon.cmon4
4: v2:10.0.1.97:3300/0 mon.cmon5 <--- the MON
being removed/added
Although it's okay to use v2 only, I'm afraid I've missed some steps and messed
the cluster up. Any advice?
Thanks again.
Best Regards,
/stwong
-----Original Message-----
From: ceph-users <ceph-users-boun...@lists.ceph.com> On Behalf Of Manuel Lausch
Sent: Tuesday, July 23, 2019 7:32 PM
To: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] Please help: change IP address of a cluster
Hi,
I had to change the IPs of my cluster some time ago. The process was quite easy.
I don't understand what you mean with configuring and deleting static routes.
The easies way is if the router allows (at least for the
change) all traffic between the old and the new network.
I did the following steps.
1. Add the new IP Network space separated to the "public network" line in your
ceph.conf
2. OSDS: stop you OSDs on the first node. Reconfigure the host network and
start your OSDs again. Repeat this for all hosts one by one
3. MON: stop and remove one mon from cluster, delete all data in
/var/ceph/mon/mon.<id> reconfigure the host network. Create the new mon
instance (don't forget the "mon host" entrys in your ceph.conf and your clients
as well) Of course this requires at least 3 Mons in your cluster!
After 2 of 5 Mons in my cluster I added the new mon adresses to my clients and
restarted them.
4. MGR: stop the mgr daemon. reconfigure the host network. Start the mgr daemon
one by one
I wouldn't recomend the "messy way" to reconfigure your mons. removing and
adding mons to the cluster is quite easy and in my opinion the most secure.
The complet IP change in our cluster worked without outage while the cluster
was in production.
I hope I could help you.
Regards
Manuel
On Fri, 19 Jul 2019 10:22:37 +0000
"ST Wong (ITSC)" <s...@itsc.cuhk.edu.hk> wrote:
> Hi all,
>
> Our cluster has to change to new IP range in same VLAN: 10.0.7.0/24
> -> 10.0.18.0/23, while IP address on private network for OSDs
> remains unchanged. I wonder if we can do that in either one following
> ways:
>
> =====================
>
> 1.
>
> a. Define static route for 10.0.18.0/23 on each node
>
> b. Do it one by one:
>
> For each monitor/mgr:
>
> - remove from cluster
>
> - change IP address
>
> - add static route to original IP range 10.0.7.0/24
>
> - delete static route for 10.0.18.0/23
>
> - add back to cluster
>
> For each OSD:
>
> - stop OSD daemons
>
> - change IP address
>
> - add static route to original IP range 10.0.7.0/24
>
> - delete static route for 10.0.18.0/23
>
> - start OSD daemons
>
> c. Clean up all static routes defined.
>
>
>
> 2.
>
> a. Export and update monmap using the messy way as described in
> http://docs.ceph.com/docs/mimic/rados/operations/add-or-rm-mons/
>
>
>
> ceph mon getmap -o {tmp}/{filename}
>
> monmaptool -rm node1 -rm node2 ... --rm node n {tmp}/{filename}
>
> monmaptool -add node1 v2:10.0.18.1:3330,v1:10.0.18.1:6789 -add node2
> v2:10.0.18.2:3330,v1:10.0.18.2:6789 ... --add nodeN
> v2:10.0.18.N:3330,v1:10.0.18.N:6789 {tmp}/{filename}
>
>
>
> b. stop entire cluster daemons and change IP addresses
>
>
> c. For each mon node: ceph-mon -I {mon-id} -inject-monmap
> {tmp}/{filename}
>
>
>
> d. Restart cluster daemons.
>
>
>
> 3. Or any better method...
> =====================
>
> Would anyone please help? Thanks a lot.
> Rgds
> /st wong
>
--
Manuel Lausch
Systemadministrator
Storage Services
1&1 Mail & Media Development & Technology GmbH | Brauerstraße 48 |
76135 Karlsruhe | Germany Phone: +49 721 91374-1847
E-Mail: manuel.lau...@1und1.de | Web: www.1und1.de
Hauptsitz Montabaur, Amtsgericht Montabaur, HRB 5452
Geschäftsführer: Alexander Charles, Thomas Ludwig, Jan Oetjen, Sascha Vollmer
Member of United Internet
Diese E-Mail kann vertrauliche und/oder gesetzlich geschützte Informationen
enthalten. Wenn Sie nicht der bestimmungsgemäße Adressat sind oder diese E-Mail
irrtümlich erhalten haben, unterrichten Sie bitte den Absender und vernichten
Sie diese E-Mail. Anderen als dem bestimmungsgemäßen Adressaten ist untersagt,
diese E-Mail zu speichern, weiterzuleiten oder ihren Inhalt auf welche Weise
auch immer zu verwenden.
This e-mail may contain confidential and/or privileged information. If you are
not the intended recipient of this e-mail, you are hereby notified that saving,
distribution or use of the content of this e-mail in any way is prohibited. If
you have received this e-mail in error, please notify the sender and delete the
e-mail.
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
