On Tue, May 9, 2017 at 4:40 PM, Brett Niver <bni...@redhat.com> wrote:
> What is your workload like? Do you have a single or multiple active > MDS ranks configured? User traffic is heavy. I can't really say in terms of mb/s or iops but it's an email server with 25k+ users, usually about 6k simultaneously connected receiving and reading emails. I have only one active MDS configured. The others are Stand-by. On Tue, May 9, 2017 at 7:18 PM, Wido den Hollander <w...@42on.com> wrote: > > > Op 9 mei 2017 om 20:26 schreef Brady Deetz <bde...@gmail.com>: > > > > > > If I'm reading your cluster diagram correctly, I'm seeing a 1gbps > > interconnect, presumably cat6. Due to the additional latency of > performing > > metadata operations, I could see cephfs performing at those speeds. Are > you > > using jumbo frames? Also are you routing? > > > > If you're routing, the router will introduce additional latency that an > l2 > > network wouldn't experience. > > > > Partially true. I am running various Ceph clusters using L3 routing and > with a decent router the latency for routing a packet is minimal, like 0.02 > ms or so. > > Ceph spends much more time in the CPU then it will take the network to > forward that IP-packet. > > I wouldn't be too afraid to run Ceph over a L3 network. > > Wido > > > On May 9, 2017 12:01 PM, "Webert de Souza Lima" <webert.b...@gmail.com> > > wrote: > > > > > Hello all, > > > > > > I'm been using cephfs for a while but never really evaluated its > > > performance. > > > As I put up a new ceph cluster, I though that I should run a benchmark > to > > > see if I'm going the right way. > > > > > > By the results I got, I see that RBD performs *a lot* better in > > > comparison to cephfs. > > > > > > The cluster is like this: > > > - 2 hosts with one SSD OSD each. > > > this hosts have 2 pools: cephfs_metadata and cephfs_cache (for > > > cache tiering). > > > - 3 hosts with 5 HDD OSDs each. > > > this hosts have 1 pool: cephfs_data. > > > > > > all details, cluster set up and results can be seen here: > > > https://justpaste.it/167fr > > > > > > I created the RBD pools the same way as the CEPHFS pools except for the > > > number of PGs in the data pool. > > > > > > I wonder why that difference or if I'm doing something wrong. > > > > > > Regards, > > > > > > Webert Lima > > > DevOps Engineer at MAV Tecnologia > > > *Belo Horizonte - Brasil* > > > > > > _______________________________________________ > > > ceph-users mailing list > > > ceph-users@lists.ceph.com > > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > _______________________________________________ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
