Thanks a lot for the explanation. It makes the most sense.
After testing using the FIO and ETHTOOL tools I discovered that my problem
was the network interface. After replacing it I managed to reach the mark
of 1150 Megabytes per second.
Em seg., 26 de set. de 2022 às 03:58, Janne Johansson
escreveu:
> Den lör 24 sep. 2022 kl 23:38 skrev Murilo Morais :
> > I'm relatively new to Ceph. I set up a small cluster with two hosts with
> 12
> > disks each host, all 3 TB SAS 7500 RPM and two 10 Gigabit interfaces. I
> > created a pool in replicated mode and configured it to use two replicas.
> >
> > What I'm finding strange is that, with these configurations, using the
> NFS
> > or iSCSI protocol, I'm getting approximately 500 Megabytes per second in
> > writing and 250 Megabytes per second in reading. Is this correct? Just
> one
> > disk can achieve rates of approximately 200 Megabytes per second.
> According
> > to Grafana's graphics, disks are not used extensively.
>
> This is very dependent on how you are testing reads. If your program*
> sends a request to read X MB, then waits for this to arrive, and then
> asks for X MB more and repeats until all of the data is received, you
> should get single-drive-speed values for reads, as long as the data is
> not in any cache.
>
> In order to make all 24 drives help out, you would need to make sure
> there is 24+ threads separately asking for data in different places to
> be read at the same time, so that the requests have a decent chance to
> go to all drives (this is slightly dependant on how data is randomized
> on the drives and hosts of course), otherwise you are mostly reading
> from one drive at a time and getting performance accordingly.
>
> Writes get more chances to "lie" a bit and tell you it is
> almost-certainly-written so that the writer thread can move on to next
> piece, and this can be done at many levels in the client, in the
> network protocols, server device drivers, drive caches and all that,
> adding up to more perf than reads would give where uncached data
> request can't be finished early.
>
> Then ceph has some overhead, and translating it to iSCSI or NFS would
> add some more overhead.
>
> *) Like you would get when running "dd if=./big-piece-of-data
> of=/dev/null bs=1M"
>
> --
> May the most significant bit of your life be positive.
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
