I'd like to add that, from all tests I did, the writing of new files only go directly to the cache tier if you set hit set count = 0.
Em Seg, 5 de jun de 2017 23:26, TYLin <wooer...@gmail.com> escreveu: > On Jun 5, 2017, at 6:47 PM, Christian Balzer <ch...@gol.com> wrote: > > Personally I avoid odd numbered releases, but my needs for stability > and low update frequency seem to be far off the scale for "normal" Ceph > users. > > W/o precise numbers of files and the size of your SSDs (which type?) it is > hard to say, but you're likely to be better off just having all metadata > on an SSD pool instead of cache-tiering. > 800MB/s sounds about right for your network and cluster in general (no > telling for sure w/o SSD/HDD details of course). > > As I pointed out before and will try to explain again below, that speed > difference, while pretty daunting, isn't all that surprising. > > > SSD: Intel S3520 240GB > HDD: WDC WD5003ABYZ-011FA0 500GB > fio: bs=4m iodepth=32 > dd: bs=4m > The test file is 20GB. > > No, not quite. Re-read what I wrote, there's a difference between RADOS > object creation and actual data (contents). > > The devs or other people with more code familiarity will correct me, but > essentially as I understand it this happens when a new RADOS object gets > created in conjunction with a cache-tier: > > 1. Client (cephfs, rbd, whatever) talks to the cache-tier and the > transaction causes a new object to be created. > Since the tier is an overlay of the actual backing storage, the object > (but not necessarily the curent data in it) needs to exist on both. > 2. Object gets created on backing storage which involves creating the > file (at zero length), any needed directories above and the entry in the > OMAP leveldb. All on HDDs, all slow. > I'm pretty sure this needs to be done and finished before the object is > usable, no journals to speed this up. > 3. Cache-tier pseudo-promotes the new object (it is empty after all) and > starts accepting writes. > > This is leaving out any metadata stuff CephFS needs to do for new "blocks" > and files, which may also be more involved than overwrites. > > Christian > > > You make it clear to me! thanks! Really appreciate your kind explanation. > > Thanks, > Ting Yi Lin > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com