On 07/03/2014 03:11 PM, VELARTIS Philipp Dürhammer wrote:
Hi,
I have a ceph cluster setup (with 45 sata disk journal on disks) and get
only 450mb/sec writes seq (maximum playing around with threads in rados
bench) with replica of 2
How many threads?
Which is about ~20Mb writes per disk (what y see in atop also)
theoretically with replica2 and having journals on disk should be 45 X
100mb (sata) / 2 (replica) / 2 (journal writes) which makes it 1125
satas in reality have 120mb/sec so the theoretical output should be more.
I would expect to have between 40-50mb/sec for each sata disk
Can somebody confirm that he can reach this speed with a setup with
journals on the satas (with journals on ssd speed should be 100mb per disk)?
or does ceph only give about ¼ of the speed for a disk? (and not the ½
as expected because of journals)
Did you verify how much each machine is doing? It could be that the data
is not distributed evenly and that on a certain machine the drives are
doing 50MB/sec.
My setup is 3 servers with: 2 x 2.6ghz xeons, 128gb ram 15 satas for
ceph (and ssds for system) 1 x 10gig for external traffic, 1 x 10gig for
osd traffic
with reads I can saturate the network but writes is far away. And I
would expect at least to saturate the 10gig with sequential writes also
Should be possible, but with 3 servers the data distribution might not
be optimal causing a lower write performance.
I've seen 10Gbit write performance on multiple clusters without any
problems.
Thank you
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Wido den Hollander
Ceph consultant and trainer
42on B.V.
Phone: +31 (0)20 700 9902
Skype: contact42on
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
