On 02/12/2018 19:48, Florian Haas wrote: > Hi Mark, > > just taking the liberty to follow up on this one, as I'd really like to > get to the bottom of this. > > On 28/11/2018 16:53, Florian Haas wrote: >> On 28/11/2018 15:52, Mark Nelson wrote: >>> Option("bluestore_default_buffered_read", Option::TYPE_BOOL, >>> Option::LEVEL_ADVANCED) >>> .set_default(true) >>> .set_flag(Option::FLAG_RUNTIME) >>> .set_description("Cache read results by default (unless hinted >>> NOCACHE or WONTNEED)"), >>> >>> Option("bluestore_default_buffered_write", Option::TYPE_BOOL, >>> Option::LEVEL_ADVANCED) >>> .set_default(false) >>> .set_flag(Option::FLAG_RUNTIME) >>> .set_description("Cache writes by default (unless hinted NOCACHE or >>> WONTNEED)"), >>> >>> >>> This is one area where bluestore is a lot more confusing for users that >>> filestore was. There was a lot of concern about enabling buffer cache >>> on writes by default because there's some associated overhead >>> (potentially both during writes and in the mempool thread when trimming >>> the cache). It might be worth enabling bluestore_default_buffered_write >>> and see if it helps reads. >> >> So yes this is rather counterintuitive, but I happily gave it a shot and >> the results are... more head-scratching than before. :) >> >> The output is here: http://paste.openstack.org/show/736324/ >> >> In summary: >> >> 1. Write benchmark is in the same ballpark as before (good). >> >> 2. Read benchmark *without* readahead is *way* better than before >> (splendid!) but has a weird dip down to 9K IOPS that I find >> inexplicable. Any ideas on that? >> >> 3. Read benchmark *with* readahead is still abysmal, which I also find >> rather odd. What do you think about that one? > > These two still confuse me. > > And in addition, I'm curious as to what you think of the approach to > configure OSDs with bluestore_cache_kv_ratio = .49, so that rather > than using 1%/99%/0% of cache memory for metadata/KV data/objects, the > OSDs use 1%/49%/50%. Is this sensible? I assume the default of not using > any memory to actually cache object data is there for a reason, but I am > struggling to grasp what that reason would be. Particularly since in > filestore, we always got in-memory object caching for free, via the page > cache.
Hi Mark, do you mind if I give this another poke? Cheers, Florian _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com