I saw Intel had a demo of a luminous cluster running on top of the line hardware, they used 2 OSD partitions with the best performance. I was interested that they would split them like that, and asked the demo person how they came to that number. I never got a really good answer except that it would provide better performance. So I guess this must be why.
On Mon, Feb 25, 2019 at 8:30 PM <[email protected]> wrote: > I create 2-4 RBD images sized 10GB or more with --thick-provision, then > run > > fio -ioengine=rbd -direct=1 -invalidate=1 -name=test -bs=4k -iodepth=128 > -rw=randwrite -pool=rpool -runtime=60 -rbdname=testimg > > For each of them at the same time. > > > How do you test what total 4Kb random write iops (RBD) you have? > > > > -----Original Message----- > > From: Vitaliy Filippov [mailto:[email protected]] > > Sent: 24 February 2019 17:39 > > To: David Turner > > Cc: ceph-users; 韦皓诚 > > Subject: *****SPAM***** Re: [ceph-users] Configuration about using nvme > > SSD > > > > I've tried 4x OSD on fast SAS SSDs in a test setup with only 2 such > > drives in cluster - it increased CPU consumption a lot, but total 4Kb > > random write iops (RBD) only went from ~11000 to ~22000. So it was 2x > > increase, but at a huge cost. > > > >> One thing that's worked for me to get more out of nvmes with Ceph is > >> to create multiple partitions on the nvme with an osd on each > > partition. > >> That > >> way you get more osd processes and CPU per nvme device. I've heard of > >> people using up to 4 partitions like this. > > > > -- > > With best regards, > > Vitaliy Filippov > > _______________________________________________ > > ceph-users mailing list > > [email protected] > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > [email protected] > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
