On 11/18/16 18:00, Thomas Danan wrote: > > I often read that small IO write and RBD are working better with > bigger filestore_max_sync_interval than default value. > > Default value is 5 sec and I saw many post saying they are using 30 sec. > > Also the slow request symptom is often linked to this parameter. > > My journals are 10GB ( collocated with OSD storage ) and overall > Client IO write throughput is around 500MB/s at peak time. > > I understand that the journal is flushed when it is half filled or > when it reach the filestore_max_sync_interval value. > > I guess I can safely give it a try ? just to see the impact ? > > Not sure if I can change this online ? > yes you can do it online
ceph tell osd.* injectargs --filestore-max-sync-interval=30 (ignore if it says unchangable, and use ceph daemon osd.$n config show| grep interval to verify it changed) I tried setting it to 5 and 120 and didn't notice any difference in blocked requests, only smoother IO performance. I have 16GB journals. BTW, my rebalance finished, and I guess the performance is a bit better, with load distributed a bit better, but blocked requests still happen if I use snapshot create + export-diff + delete snapshot, and make qemu clients hang the same. A 30s sleep between image backups prevents most, but not large exports (or snap creates or snap rm?). I plan to test and see if really heavy client IO can cause it too (without creating more objects, like in place modification rather than using less space like trim and using more space), or only other io (rbd commands, rebalancing, cow, scrub). So far I suspect it has to do with either poor xattr performance, or something to do with a large amount of splitting or otherwise reorganizing many pg/object directories all at once.
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
