The standard argument that it helps preventing recovery traffic from
clogging the network and impacting client traffic is missleading:

* write client traffic relies on the backend network for replication
operations: your client (write) traffic is impacted anyways if the
backend network is full
* you are usually not limited by network speed for recovery (except
for 1 gbit networks), and if you are you probably want to reduce
recovery speed anyways if you would run into that limit

Paul

-- 
Paul Emmerich

Looking for help with your Ceph cluster? Contact us at https://croit.io

croit GmbH
Freseniusstr. 31h
81247 München
www.croit.io
Tel: +49 89 1896585 90

On Wed, Apr 17, 2019 at 10:39 AM Lars Täuber <taeu...@bbaw.de> wrote:
>
> Wed, 17 Apr 2019 09:52:29 +0200
> Stefan Kooman <ste...@bit.nl> ==> Lars Täuber <taeu...@bbaw.de> :
> > Quoting Lars Täuber (taeu...@bbaw.de):
> > > > I'd probably only use the 25G network for both networks instead of
> > > > using both. Splitting the network usually doesn't help.
> > >
> > > This is something i was told to do, because a reconstruction of failed
> > > OSDs/disks would have a heavy impact on the backend network.
> >
> > Opinions vary on running "public" only versus "public" / "backend".
> > Having a separate "backend" network might lead to difficult to debug
> > issues when the "public" network is working fine, but the "backend" is
> > having issues and OSDs can't peer with each other, while the clients can
> > talk to all OSDs. You will get slow requests and OSDs marking each other
> > down while they are still running etc.
>
> This I was not aware of.
>
>
> > In your case with only 6 spinners max per server there is no way you
> > will every fill the network capacity of a 25 Gb/s network: 6 * 250 MB/s
> > (for large spinners) should be just enough to fill a 10 Gb/s link. A
> > redundant 25 Gb/s link would provide 50 Gb/s of bandwith, enough for
> > both OSD replication traffic and client IO.
>
> The reason for the choice for the 25GBit network was because a remark of 
> someone, that the latency in this ethernet is way below that of 10GBit. I 
> never double checked this.
>
>
> >
> > My 2 cents,
> >
> > Gr. Stefan
> >
>
> Cheers,
> Lars
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Reply via email to