On Mon, May 16, 2016 at 12:20 PM, Nick Fisk <n...@fisk.me.uk> wrote: > > > > -----Original Message----- > > From: Peter Kerdisle [mailto:peter.kerdi...@gmail.com] > > Sent: 16 May 2016 11:04 > > To: Nick Fisk <n...@fisk.me.uk> > > Cc: ceph-users@lists.ceph.com > > Subject: Re: [ceph-users] Erasure pool performance expectations > > > > > > On Mon, May 16, 2016 at 11:58 AM, Nick Fisk <n...@fisk.me.uk> wrote: > > > -----Original Message----- > > > From: Peter Kerdisle [mailto:peter.kerdi...@gmail.com] > > > Sent: 16 May 2016 10:39 > > > To: n...@fisk.me.uk > > > Cc: ceph-users@lists.ceph.com > > > Subject: Re: [ceph-users] Erasure pool performance expectations > > > > > > I'm forcing a flush by lower the cache_target_dirty_ratio to a lower > value. > > > This forces writes to the EC pool, these are the operations I'm trying > to > > > throttle a bit. I am understanding you correctly that's throttling > only works > > for > > > the other way around? Promoting cold objects into the hot cache? > > > > Yes that’s correct. You want to throttle the flushes which is done by > another > > setting(s) > > > > Firstly set something like this in your ceph.conf > > osd_agent_max_low_ops = 1 > > osd_agent_max_ops = 4 > > I did not know about this, that's great, I will play around with these. > > > > > > This controls how many parallel threads the tiering agent will use. You > can > > bump them up later if needed. > > > > Next set on your cache pools, these two settings. Try and keep them about > > .2 apart. So something like .4 and .6 are good to start with. > > cache_target_dirty_ratio > > cache_target_dirty_high_ratio > > Here is actually the heart of the matter. Ideally I would love to run it > at 0.0 if > > that makes sense. I want no dirty objects in my hot cache at all, has > anybody > > ever tried this? Right now I'm just pushing cache_target_dirty_ratio > during > > low activity moments by setting it to 0.2 and then bringing it back up > to 0.6 > > when it's done or activity starts up again. > > You might want to rethink that slightly. Keeping in mind that with EC > pools currently any write will force a promotion and then dirty the object. > If you are then almost immediately flushing the objects back down, you are > going to end up with a lot of amplification for writes. You want to keep > dirty objects in the cache pool so that you don't incur this penalty if > they are going to be written to again in the near future. >
After reading this again it does raise an other question. I'm not sure I understood promotions correctly before this response. A promotion is the act of moving a piece from the EC pool onto the cache tier, is this correct? If so does a promotion remove the actual data from the EC pool and make it dirty again? I was under the impression it was still on the EC pool but also on the cache tier at that point. > > I'm guessing what you want is a buffer, so that you can have bursts of > activity without incurring the performance penalty of flushing? That’s > hopefully what the high and low flushes should give you. By setting to .4 > and .6, you will have a .2x"Cache Tier Capacity" buffer of cache tier > space, where only slow flushing will occur. > This is indeed one of the reasons. The other reason was that I thought that by removing dirty objects I didn't need replication on the cache tier, which I'm now starting to doubt again...
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
