As the single job spec being referenced (from Sebastien Han’s blog<http://www.sebastien-han.fr/blog/2014/10/10/ceph-how-to-test-if-your-ssd-is-suitable-as-a-journal-device/>, as I understand) includes the use of the —sync flag, the HBA and storage bus are unlikely to dominate the test results. O_DSync operations must wait until a write is complete before starting another operation, which leaves both the CPU and the HBA ‘waiting' on the storage media for the completion.
All that said (mostly for those not familiar with the impact of —sync on SSDs): the communications path is a generally longer and more complicated with the introduction of (any) SAS HBA in this use case. SAS is great for handling multipath, multi-device communications, but for a simple point-to-point link the SATA protocol ‘wins’ out: the SATA Controller is closer to the CPU, avoids the PCIe bus, avoids Expander chips (hops) and the entire SATA->SAS->SATA tunneling operations are avoided too. Hence: using SATA SSDs on a SAS bus is a ‘sub-optimal’ solution in most cases, although 12G SAS overcomes some of this while SATA is stuck at 6G. Bottom line: for the high-level goal of getting good O_DSync performance, finding a ‘better' HBA isn’t a useful exercise. Having faster storage media and a simpler IO path (like NVMe) should yield the best results, especially for synchronous IO. My $.02. -- Paul I've checked with a MegaRaid SAS 2208, then I get ~40 MB/s, both with a 1.9TB and a 240GB SM863 model. So it seems the LSI MegaRaid HBA's is not optimized for a lot of single job IOPS...
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
