On 13/12/2018 09:53, Ashley Merrick wrote:
I have a Mimic Bluestore EC RBD Pool running on 8+2, this is currently
running across 4 node's.
3 Node's are running Toshiba disk's while one node is running Segate
disks (same size, spinning speed, enterprise disks e.t.c), I have
noticed huge difference in IOWAIT and disk latency performance between
the two sets of disks, can also be seen from a ceph osd perf during
read and write operations.
Speaking to my host (server provider), they bench marked the two disks
before approving them for use in this type of server, they actually
saw higher performance from the Toshiba disk during their tests.
They did however state there test where at higher / larger block
sizes, I can imagine CEPH using EC of 8+2 the block sizes / requests
are quite small?
Is there anything I can do ? Changing the RBD object size & stripe
unit to a bigger than default? Will this make the data sent to the
disk larger chunks at once compared to lot's of smaller block's.
If anyone else has any advice I'm open to trying.
P.s I have already disabled the disk cache on all disks and this was
causing high write latency across all.
Thanks
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Since you say there is a huge difference between the disk types under
your current workload, then i would focus on this, the logical thing to
do is to replace them. You can probably run further benchmarks with
fsync write speed at lower block sizes, but i think your current
observation is conclusive enough.
Other less recommended options: use a lower ec profile such as k4 m2,
getting a controller with write back cache. For sequential io increasing
your read_ahead_kb, using librbd client cache, adjusting your client os
cache parameters. Also if you have a controlled application like a
backup app where you can specify the block size, then increase it to
above 1MB. But again i would recommend you focus on changing disks.
/Maged
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com