Re: [ceph-users] Adding cluster network to running cluster

2018-06-07 Thread mj
On 06/07/2018 01:45 PM, Wido den Hollander wrote: Removing cluster network is enough. After the restart the OSDs will not publish a cluster network in the OSDMap anymore. You can keep the public network in ceph.conf and can even remove that after you removed the 10.10.x.x addresses from the

Re: [ceph-users] Adding cluster network to running cluster

2018-06-07 Thread Wido den Hollander
On 06/07/2018 01:39 PM, mj wrote: > Hi, > > Please allow me to ask one more question: > > We currently have a seperated network: cluster on 10.10.x.x and public > on 192.168.x.x. > > I would like to migrate all network to 192.168.x.x setup, which would > give us 2*10G. > > Is simply changing

Re: [ceph-users] Adding cluster network to running cluster

2018-06-07 Thread mj
Hi, Please allow me to ask one more question: We currently have a seperated network: cluster on 10.10.x.x and public on 192.168.x.x. I would like to migrate all network to 192.168.x.x setup, which would give us 2*10G. Is simply changing the cluster network in ceph.conf and a restart enough

Re: [ceph-users] Adding cluster network to running cluster

2018-06-07 Thread Paul Emmerich
We also build almost all of our clusters with a single Ceph network. 2x10 Gbit/s is almost never the bottleneck. Paul 2018-06-07 11:05 GMT+02:00 Wido den Hollander : > > > On 06/07/2018 10:56 AM, Kevin Olbrich wrote: > > Realy? > > > > I always thought that splitting the replication network is

Re: [ceph-users] Adding cluster network to running cluster

2018-06-07 Thread Wido den Hollander
On 06/07/2018 10:56 AM, Kevin Olbrich wrote: > Realy? > > I always thought that splitting the replication network is best practice. > Keeping everything in the same IPv6 network is much easier. > No, there is no big benefit unless your usecase (which 99% isn't) asks for it. Keep it simple, on

Re: [ceph-users] Adding cluster network to running cluster

2018-06-07 Thread Kevin Olbrich
Realy? I always thought that splitting the replication network is best practice. Keeping everything in the same IPv6 network is much easier. Thank you. Kevin 2018-06-07 10:44 GMT+02:00 Wido den Hollander : > > > On 06/07/2018 09:46 AM, Kevin Olbrich wrote: > > Hi! > > > > When we installed our

Re: [ceph-users] Adding cluster network to running cluster

2018-06-07 Thread Wido den Hollander
On 06/07/2018 09:46 AM, Kevin Olbrich wrote: > Hi! > > When we installed our new luminous cluster, we had issues with the > cluster network (setup of mon's failed). > We moved on with a single network setup. > > Now I would like to set the cluster network again but the cluster is in > use (4 no

Re: [ceph-users] Adding cluster network to running cluster

2018-06-07 Thread Burkhard Linke
Hi, I may be wrong, but AFAIK the cluster network is only used to bind the corresponding functionality to the correct network interface. There's no check for a common CIDR range or something similar in CEPH. As long as the traffic is routeable from the current public network and the new cl

Re: [ceph-users] Adding cluster network to running cluster

2018-06-07 Thread David Turner
2 Things about this. First, Mons do not communicate over the cluster network. Only OSD daemons send any traffic over that network. Mons, MDS, RGW, MGR, etc all communicate over the public network. OSDs communicate with all clients, Mons, etc on the public network and with each other on the clus

[ceph-users] Adding cluster network to running cluster

2018-06-07 Thread Kevin Olbrich
Hi! When we installed our new luminous cluster, we had issues with the cluster network (setup of mon's failed). We moved on with a single network setup. Now I would like to set the cluster network again but the cluster is in use (4 nodes, 2 pools, VMs). What happens if I set the cluster network o