Yes, linear io speed was concern during benchmark. I can not predict how
much linear IO would be generated by clients (compare to IOPS) so we
going to balance HDD-OSD per SSD according to real usage. If users would
generate too much random IO, we will raise HDD/SSD ratio, if they would
generate more linear-write load, we will reduce that number. I plan to
do it by reserving space for 'more HDD' or 'more SSD' in planned servers
- they will go to production with ~50% slot utilization.
My main concern is that random IO for OSD includes not only writes, but
reads too, and cold random read will slower HDD performance
significantly. On my previous experience, any weekly cronjob on server
with backup (or just 'find /') cause bad spikes of cold read, and that
drastically diminish HDD performance.
(TL;DR; I don't belive we ever would have conditions when HDD can give
few dozens of MB/s of writing).
Thank you for advice.
On 07/12/2016 04:03 PM, Vincent Godin wrote:
Hello.
I've been testing Intel 3500 as journal store for few HDD-based OSD. I
stumble on issues with multiple partitions (>4) and UDEV (sda5, sda6,etc
sometime do not appear after partition creation). And I'm thinking that
partition is not that useful for OSD management, because linux do no
allow partition rereading with it contains used volumes.
So my question: How you store many journals on SSD? My initial thoughts:
1) filesystem with filebased journals
2) LVM with volumes
Anything else? Best practice?
P.S. I've done benchmarking: 3500 can support up to 16 10k-RPM HDD.
Hello,
I would like to advertise you not using 1 SSD for 16 HDD. Ceph journal
is not only a journal but a write cache during operation. I had that
kind of configuration with 1 SSD for 20 SATA HDD. With a Ceph bench,
i notice that my rate whas limited between 350 and 400 MB/s. In fact,
a iostat show me that my SSD was 100% utilised with a rate of 350-400
MB/s.
If you consider that a SATA HDD can have a max average rate of 100
MB/s, you need to configure one SSD (which can rate till 400 MB/s) for
4 SATA HDD
Vincent
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com