Thanks a lot for the explanation. It makes the most sense. After testing using the FIO and ETHTOOL tools I discovered that my problem was the network interface. After replacing it I managed to reach the mark of 1150 Megabytes per second.
Em seg., 26 de set. de 2022 às 03:58, Janne Johansson <icepic...@gmail.com> escreveu: > Den lör 24 sep. 2022 kl 23:38 skrev Murilo Morais <mur...@evocorp.com.br>: > > I'm relatively new to Ceph. I set up a small cluster with two hosts with > 12 > > disks each host, all 3 TB SAS 7500 RPM and two 10 Gigabit interfaces. I > > created a pool in replicated mode and configured it to use two replicas. > > > > What I'm finding strange is that, with these configurations, using the > NFS > > or iSCSI protocol, I'm getting approximately 500 Megabytes per second in > > writing and 250 Megabytes per second in reading. Is this correct? Just > one > > disk can achieve rates of approximately 200 Megabytes per second. > According > > to Grafana's graphics, disks are not used extensively. > > This is very dependent on how you are testing reads. If your program* > sends a request to read X MB, then waits for this to arrive, and then > asks for X MB more and repeats until all of the data is received, you > should get single-drive-speed values for reads, as long as the data is > not in any cache. > > In order to make all 24 drives help out, you would need to make sure > there is 24+ threads separately asking for data in different places to > be read at the same time, so that the requests have a decent chance to > go to all drives (this is slightly dependant on how data is randomized > on the drives and hosts of course), otherwise you are mostly reading > from one drive at a time and getting performance accordingly. > > Writes get more chances to "lie" a bit and tell you it is > almost-certainly-written so that the writer thread can move on to next > piece, and this can be done at many levels in the client, in the > network protocols, server device drivers, drive caches and all that, > adding up to more perf than reads would give where uncached data > request can't be finished early. > > Then ceph has some overhead, and translating it to iSCSI or NFS would > add some more overhead. > > *) Like you would get when running "dd if=./big-piece-of-data > of=/dev/null bs=1M" > > -- > May the most significant bit of your life be positive. > _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
