On Tue, Jun 4, 2019 at 1:39 PM <vita...@yourcmc.ru> wrote: > >> Basically they max out at around 1000 IOPS and report 100% > >> utilization and feel slow. > >> > >> Haven't seen the 5200 yet. > > Micron 5100s performs wonderfully! > > You have to just turn its write cache off: > > hdparm -W 0 /dev/sdX > > 1000 IOPS means you haven't done it. Although even with write cache > enabled I observe like ~5000 iops, not 1000, but that delta is probably > just eaten by Ceph :)) >
can confirm that there are several disks where the write cache seems to be broken and this helps a lot. Good to know that this is one of these disks. (The cluster with these disks that I've worked on didn't need more than a few IOPS per disk and had other problems, so I didn't check it there) Paul > > With write cache turned off 5100 is capable of up to 40000 write iops. > 5200 is slightly worse, but only slightly: it still gives ~25000 iops. > > Funny thing is that the same applies to a lot of server SSDs with > supercapacitors. As I understand when their write cache is turned on > every `fsync` is translated to SATA FLUSH CACHE, and the latter is > interpreted by the drive as "please flush all caches, including > capacitor-protected write cache". > > And when you turn it off the drive just writes at its full speed and > doesn't flush the cache because it has capacitors to account for a > possible power loss. > > You don't need to disable cache explicitly only with some HBAs that do > it internally. >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
